LangChain
¶
Bases: BaseLLM
Using chains in langchain to generate keywords.
Currently, only chains from question answering is implemented. See: https://langchain.readthedocs.io/en/latest/modules/chains/combine_docs_examples/question_answering.html
NOTE: The resulting keywords are expected to be separated by commas so any changes to the prompt will have to make sure that the resulting keywords are comma-separated.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
chain
|
A langchain chain that has two input parameters, |
required | |
prompt
|
str
|
The prompt to be used in the model. If no prompt is given,
|
None
|
verbose
|
bool
|
Set this to True if you want to see a progress bar for the keyword extraction. |
False
|
Usage:
To use this, you will need to install the langchain package first. Additionally, you will need an underlying LLM to support langchain, like openai:
pip install langchain
pip install openai
Then, you can create your chain as follows:
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
chain = load_qa_chain(OpenAI(temperature=0, openai_api_key=my_openai_api_key), chain_type="stuff")
Finally, you can pass the chain to KeyBERT as follows:
from keybert.llm import LangChain
from keybert import KeyLLM
# Create your LLM
llm = LangChain(chain)
# Load it in KeyLLM
kw_model = KeyLLM(llm)
# Extract keywords
document = "The website mentions that it only takes a couple of days to deliver but I still have not received mine."
keywords = kw_model.extract_keywords(document)
You can also use a custom prompt:
prompt = "What are these documents about? Please give a single label."
llm = LangChain(chain, prompt=prompt)
Source code in keybert\llm\_langchain.py
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
|
extract_keywords(documents, candidate_keywords=None)
¶
Extract topics.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
documents
|
List[str]
|
The documents to extract keywords from |
required |
candidate_keywords
|
List[List[str]]
|
A list of candidate keywords that the LLM will fine-tune For example, it will create a nicer representation of the candidate keywords, remove redundant keywords, or shorten them depending on the input prompt. |
None
|
Returns:
Name | Type | Description |
---|---|---|
all_keywords |
All keywords for each document |
Source code in keybert\llm\_langchain.py
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
|