Langchain-GenrativeAI-Series GitHub Details, Stars and Alternatives | OpenRepoFinder
Warishayat / repository
Langchain-GenrativeAI-Series
This series focuses on exploring LangChain and generative AI, providing practical guides and tutorials for building advanced AI applications. Each part covers key concepts, tools, and techniques to help you leverage LangChain for creating powerful, data-driven solutions. Ideal for developers looking to dive into AI and NLP development.
A transparent discovery signal based on current public GitHub metadata.
Recent activity35% weight
10
Community adoption25% weight
9
Maintenance state20% weight
100
License clarity10% weight
100
Project information10% weight
35
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
README preview
LangChain for GenAI & NLP
Welcome to my project! This repository showcases my journey and learnings as I explore LangChain, Generative AI, and Natural Language Processing (NLP). Through this repository, I will share my work on building powerful chains, prompt templates, embeddings, memory management, multi-agent systems, LangGraph, and more, as I continue to delve into the world of modern AI. Follow me as I continue to explore new tools, algorithms, and techniques!
🚀 Starting My Chain Here
I’m excited to start creating a variety of chains using LangChain, an open-source framework designed to make it easier to work with large language models (LLMs) and connect them with external data sources. Here's what I've learned so far:
Key Topics I've Explored:
Prompt Templates: Using LangChain’s ChatPromptTemplate and other prompt tools to create dynamic, reusable templates for interacting with LLMs.
Output Parsers: Parsing outputs from language models to make them more structured and useful for downstream tasks.
Document Loaders: Efficiently loading and handling external documents for processing with LLMs.
Text Splitters: Breaking text into smaller, manageable chunks using CharacterTextSplitter and RecursiveTextSplitter to improve processing speed and accuracy.
Embeddings: Integrating embeddings from HuggingFace, Google Generative GenAI, and other sources for powerful document retrieval and search capabilities.
Vector Stores: Storing and retrieving vector embeddings efficiently using tools like FAISS, Chroma, Pinecone, and others for high-performance document retrieval and similarity search.
Contextual Compression: Leveraging contextual compression techniques to optimize input and output data for LLMs, ensuring more efficient processing and delivering state-of-the-art results in natural language understanding and generation tasks.
LLMChain: Using LangChain’s LLMChain to build powerful, modular chains that connect multiple LLM calls, transforming workflows into multi-step processes.
Memory: Implementing Memory features like WindowBufferMemory to manage long-term interactions, store information, and maintain context across sessions.
VectorStoreRetriever: Using LangChain’s VectorStoreRetriever to perform efficient document retrieval using vector embeddings from a vector store.
ALGORITHMICALLY RELATED
Similar Open-Source Projects
Selected from shared topics, language and repository description—not editorial ratings.
Part I - WeatherPy In this example, you’ll be creating a Python script to visualize the weather of 500+ cities across the world of varying distance from the equator. To accomplish this, you’ll be utilizing a simple Python library, the OpenWeatherMap API, and a little common sense to create a representative model of weather across world cities. Your first objective is to build a series of scatter plots to showcase the following relationships: Temperature (F) vs. Latitude Humidity (%) vs. Latitude Cloudiness (%) vs. Latitude Wind Speed (mph) vs. Latitude After each plot add a sentence or too explaining what the code is and analyzing. Your next objective is to run linear regression on each relationship, only this time separating them into Northern Hemisphere (greater than or equal to 0 degrees latitude) and Southern Hemisphere (less than 0 degrees latitude): Northern Hemisphere - Temperature (F) vs. Latitude Southern Hemisphere - Temperature (F) vs. Latitude Northern Hemisphere - Humidity (%) vs. Latitude Southern Hemisphere - Humidity (%) vs. Latitude Northern Hemisphere - Cloudiness (%) vs. Latitude Southern Hemisphere - Cloudiness (%) vs. Latitude Northern Hemisphere - Wind Speed (mph) vs. Latitude Southern Hemisphere - Wind Speed (mph) vs. Latitude After each pair of plots explain what the linear regression is modelling such as any relationships you notice and any other analysis you may have. Your final notebook must: Randomly select at least 500 unique (non-repeat) cities based on latitude and longitude. Perform a weather check on each of the cities using a series of successive API calls. Include a print log of each city as it’s being processed with the city number and city name. Save a CSV of all retrieved data and a PNG image for each scatter plot. Part II - VacationPy Now let’s use your skills in working with weather data to plan future vacations. Use jupyter-gmaps and the Google Places API for this part of the assignment. Note: if you having trouble displaying the maps try running jupyter nbextension enable --py gmaps in your environment and retry. Create a heat map that displays the humidity for every city from the part I of the homework. heatmap Narrow down the DataFrame to find your ideal weather condition. For example: A max temperature lower than 80 degrees but higher than 70. Wind speed less than 10 mph. Zero cloudiness. Drop any rows that don’t contain all three conditions. You want to be sure the weather is ideal. Note: Feel free to adjust to your specifications but be sure to limit the number of rows returned by your API requests to a reasonable number. Using Google Places API to find the first hotel for each city located within 5000 meters of your coordinates. Plot the hotels on top of the humidity heatmap with each pin containing the Hotel Name, City, and Country. hotel map As final considerations: Create a new GitHub repository for this project called API-Challenge (note the kebab-case). Do not add to an existing repo You must complete your analysis using a Jupyter notebook. You must use the Matplotlib or Pandas plotting libraries. For Part I, you must include a written description of three observable trends based on the data. You must use proper labeling of your plots, including aspects like: Plot Titles (with date of analysis) and Axes Labels. For max intensity in the heat map, try setting it to the highest humidity found in the data set. Hints and Considerations The city data you generate is based on random coordinates as well as different query times; as such, your outputs will not be an exact match to the provided starter notebook. You may want to start this assignment by refreshing yourself on the geographic coordinate system. Next, spend the requisite time necessary to study the OpenWeatherMap API. Based on your initial study, you should be able to answer basic questions about the API: Where do you request the API key? Which Weather API in particular will you need? What URL endpoints does it expect? What JSON structure does it respond with? Before you write a line of code, you should be aiming to have a crystal clear understanding of your intended outcome. A starter code for Citipy has been provided. However, if you’re craving an extra challenge, push yourself to learn how it works: citipy Python library. Before you try to incorporate the library into your analysis, start by creating simple test cases outside your main script to confirm that you are using it correctly. Too often, when introduced to a new library, students get bogged down by the most minor of errors – spending hours investigating their entire code – when, in fact, a simple and focused test would have shown their basic utilization of the library was wrong from the start. Don’t let this be you! Part of our expectation in this challenge is that you will use critical thinking skills to understand how and why we’re recommending the tools we are. What is Citipy for? Why would you use it in conjunction with the OpenWeatherMap API? How would you do so? In building your script, pay attention to the cities you are using in your query pool. Are you getting coverage of the full gamut of latitudes and longitudes? Or are you simply choosing 500 cities concentrated in one region of the world? Even if you were a geographic genius, simply rattling 500 cities based on your human selection would create a biased dataset. Be thinking of how you should counter this. (Hint: Consider the full range of latitudes). Once you have computed the linear regression for one chart, the process will be similar for all others. As a bonus, try to create a function that will create these charts based on different parameters. Remember that each coordinate will trigger a separate call to the Google API. If you’re creating your own criteria to plan your vacation, try to reduce the results in your DataFrame to 10 or fewer cities. Lastly, remember – this is a challenging activity. Push yourself! If you complete this task, then you can safely say that you’ve gained a strong mastery of the core foundations of data analytics and it will only go better from here. Good luck!
Agents & Multi-Agents: Building intelligent agents that can autonomously make decisions, reason about tasks, and even use external tools to complete complex workflows.
LangGraph: Creating and visualizing AI workflows with LangChain’s LangGraph, which helps structure multi-step processes and visualize the relationships between tasks.
🔧 Technologies & Tools Used:
LangChain: A framework for developing applications powered by LLMs.
HuggingFace Embeddings: Pre-trained models for converting text into embeddings for document retrieval.
Google Generative GenAI: Leveraging Google's advanced generative models for high-quality text generation and understanding.
Text Splitters: Using advanced text splitting techniques to break down long documents into more digestible chunks.
Prompt Templates: Dynamically creating prompts to interact with LLMs in specific ways.
Vector Stores: Using FAISS, Chroma, Pinecone, and other vector stores to manage and search through high-dimensional embeddings efficiently.
Contextual Compression: Techniques to compress large inputs while retaining context, optimizing performance and achieving state-of-the-art results in natural language generation tasks.
LLMChain: Creating multi-step workflows and integrating LLMs to build chains that combine multiple tasks or LLM calls.
Memory: Managing session-based memory using tools like WindowBufferMemory to track interactions over time.
VectorStoreRetriever: Efficiently retrieving documents or information using vector embeddings stored in a vector store.
Agents: Building intelligent agents that can reason about tasks and autonomously interact with external APIs and systems.
Multi-Agents: Managing multiple agents working together to complete complex workflows or tasks in parallel.
LangGraph: Visualizing and managing complex AI workflows and task dependencies using the LangGraph module.
📝 Key Features & Learnings
1. Prompt Templates
Templates define how the input data is structured and guide the language model's behavior.
Example: Using a chat-based prompt to build a dynamic conversation flow.
2. Output Parsers
After generating a response, it's important to parse and structure the output.
Example: Parsing text for specific pieces of information such as names, dates, or key concepts.
3. Document Loaders
Load documents from various sources (e.g., PDFs, CSVs) and make them ready for processing by the model.
Example: Using document loaders to bring in knowledge from multiple sources for retrieval and generation.
4. Text Splitters
CharacterTextSplitter: Splits large documents into smaller text blocks by a specific number of characters.
RecursiveTextSplitter: Splits documents in a more recursive manner based on semantic boundaries like paragraphs and headings.
5. Embeddings
HuggingFace Embeddings: Transform text data into vector embeddings for search and retrieval tasks.
Google Generative GenAI Embeddings: Integrating Google's GenAI embeddings to enhance text generation and retrieval capabilities.
6. Vector Stores
FAISS: A popular library for efficient similarity search and clustering of embeddings, widely used for large-scale document retrieval tasks.
Chroma: An open-source vector database for storing and querying embeddings that integrates with LangChain to facilitate powerful search and retrieval.
Pinecone: A managed vector database solution that enables high-performance, real-time similarity search and indexing of vector embeddings.
Weaviate: A vector search engine for machine learning models that also integrates well with LangChain for document retrieval and other use cases.
7. Contextual Compression
Contextual Compression involves using techniques that compress the input data while retaining the most important contextual information. This method allows large documents or inputs to be handled efficiently by LLMs without losing key information, improving the model’s processing speed and delivering high-quality outputs. By applying contextual compression, I aim to optimize the interaction with language models, achieving state-of-the-art performance for NLP tasks.
8. LLMChain
The LLMChain module helps build complex workflows by chaining multiple LLM calls together. This allows for modular, reusable components in your NLP pipeline, making it easier to handle tasks like document generation, summarization, and more.
9. Memory (WindowBufferMemory)
Memory is essential for maintaining context across interactions. I’ve explored using WindowBufferMemory to manage the model’s memory, enabling it to recall past interactions or outputs and provide more context-aware responses over time.
10. VectorStoreRetriever
VectorStoreRetriever is used to retrieve relevant documents or pieces of information from a vector store based on their semantic similarity to a given query. This tool is essential for building document-based applications where context retrieval is key.
11. Agents & Multi-Agents
Agents in LangChain allow models to autonomously interact with external tools or APIs to achieve a goal. I’ve experimented with using agents to handle tasks like querying external data or processing information across multiple steps.
Multi-Agents take this concept further, enabling the coordination of multiple agents working in parallel or sequentially to accomplish more complex goals. This is ideal for scenarios where tasks can be divided and worked on simultaneously, speeding up the process and improving efficiency.
12. LangGraph
LangGraph is a visualization tool within LangChain that helps design and manage complex workflows. It allows users to graphically represent the steps and dependencies in a chain, providing clarity on how tasks are related. This makes it easier to debug and optimize workflows as they grow in complexity.
🧑💻 Code Snippets & Examples
Example 1: Creating a Prompt Template
from langchain.prompts import ChatPromptTemplate
# Define a simple chat-based prompt template
chat_prompt = ChatPromptTemplate.from_messages([
("ai", "You are a helpful assistant. Based on the user's question '{question}', you will provide an answer."),
("user", "{question}")
])
formatted_prompt = chat_prompt.format_messages(question="Who is Elon Musk?")
Example 2: Loading Documents and Embeddings
from langchain_huggingface.embeddings import HuggingFaceEmbeddings
def embeddings(data):
if not isinstance(data, str):
data = str(data)
embeddings = HuggingFaceEmbeddings()
vector = embeddings.embed_query(data)
return vector
# Example data and embeddings
data = "Information about Elon Musk"
result_vectors = embeddings(data)
Example 3: Using a Text Splitter
from langchain.text_splitter import CharacterTextSplitter, RecursiveCharacterTextSplitter
# Split a long document into smaller chunks
text = "This is a very long document that needs to be split into smaller chunks."
splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=20)
chunks = splitter.split_text(text)
print(chunks)
Example 4: Using a Vector Store (FAISS) for Document Retrieval
import faiss
import numpy as np
# Create random embeddings for 100 documents
embeddings = np.random.random((100, 128)).astype('float32')
# Create a FAISS index and add the embeddings
index = faiss.IndexFlatL2(128) # Use L2 distance for search
index.add(embeddings)
# Now, query with a new embedding (for example, from a user's question)
query = np.random.random((1, 128)).astype('float32')
distances, indices = index.search(query, k=5)
print("Closest matches:", indices)
print("Distances:", distances)
Example 5: Using an Agent to Query External Tools
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
tools = [
Tool(
name="calculator",
func=my_calculation_function, # Define this function to use the tool
description="Performs mathematical operations"
)
]
agent = initialize_agent(tools, AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
response = agent.run("What is 3 + 5?")
print(response)
📈 What’s Coming Next
This is just the beginning of my journey with LangChain, and there's much more to explore! Some exciting topics I plan to cover next include:
Advanced document retrieval techniques
Building multi-step chains for more complex workflows
Integrating with external APIs for real-time data access
Fine-tuning language models on custom datasets
Exploring multi-agent coordination in real-world scenarios
Designing complex workflows with LangGraph
🌟 Follow Me on GitHub!
If you find this project interesting and want to stay updated with my progress, feel free to follow me on GitHub! I’ll be pushing more code, tutorials, and ideas as I continue exploring LangChain and its potential.
This repository explores **Retrieval-Augmented Generation (RAG)** in depth, covering both foundational concepts and advanced agentic architectures. The project is organized as a **progressive series of Jupyter notebooks**, each focusing on a specific RAG component, from query transformation to adaptive agents and knowledge graph integration.
## WeatherPy In this example, you'll be creating a Python script to visualize the weather of 500+ cities across the world of varying distance from the equator. To accomplish this, you'll be utilizing a [simple Python library](https://pypi.python.org/pypi/citipy), the [OpenWeatherMap API](https://openweathermap.org/api), and a little common sense to create a representative model of weather across world cities. Your objective is to build a series of scatter plots to showcase the following relationships: * Temperature (F) vs. Latitude * Humidity (%) vs. Latitude * Cloudiness (%) vs. Latitude * Wind Speed (mph) vs. Latitude Your final notebook must: * Randomly select **at least** 500 unique (non-repeat) cities based on latitude and longitude. * Perform a weather check on each of the cities using a series of successive API calls. * Include a print log of each city as it's being processed with the city number and city name. * Save both a CSV of all data retrieved and png images for each scatter plot. As final considerations: * You must complete your analysis using a Jupyter notebook. * You must use the Matplotlib or Pandas plotting libraries. * You must include a written description of three observable trends based on the data. * You must use proper labeling of your plots, including aspects like: Plot Titles (with date of analysis) and Axes Labels. * See [Example Solution](WeatherPy_Example.pdf) for a reference on expected format. ## Hints and Considerations * You may want to start this assignment by refreshing yourself on the [geographic coordinate system](http://desktop.arcgis.com/en/arcmap/10.3/guide-books/map-projections/about-geographic-coordinate-systems.htm). * Next, spend the requisite time necessary to study the OpenWeatherMap API. Based on your initial study, you should be able to answer basic questions about the API: Where do you request the API key? Which Weather API in particular will you need? What URL endpoints does it expect? What JSON structure does it respond with? Before you write a line of code, you should be aiming to have a crystal clear understanding of your intended outcome. * A starter code for Citipy has been provided. However, if you're craving an extra challenge, push yourself to learn how it works: [citipy Python library](https://pypi.python.org/pypi/citipy). Before you try to incorporate the library into your analysis, start by creating simple test cases outside your main script to confirm that you are using it correctly. Too often, when introduced to a new library, students get bogged down by the most minor of errors -- spending hours investigating their entire code -- when, in fact, a simple and focused test would have shown their basic utilization of the library was wrong from the start. Don't let this be you! * Part of our expectation in this challenge is that you will use critical thinking skills to understand how and why we're recommending the tools we are. What is Citipy for? Why would you use it in conjunction with the OpenWeatherMap API? How would you do so? * In building your script, pay att
Welcome to the repository dedicated to in-depth financial stock time series analysis! In this dynamic collection, we explore the fascinating world of stock market data through Jupyter Notebook (ipynb) files. The initial focus is on Autoregressive (AR) modeling, offering valuable insights into stock price prediction.
This Jupyter Notebook explores machine learning techniques for time series analysis, including CNNs, Transformers, and LSTMs. It covers model training and evaluation, with a focus on forecasting and pattern recognition. The models are implemented on a dataset which contains accelerometer data from Pebble smartwatches for 6 distinct gestures.