If you’ve recently set up a local installation of Llama, you might be eager to test if everything is working as expected. This blog post will walk you through a simple Python program to validate your installation by sending a test query to the locally running Llama server.
Why Test Your Local Installation?
Running a language model like Llama locally is exciting because it gives you more control over your data and reduces reliance on external services. However, to ensure your setup is functional, it’s always a good idea to run a quick test. The Python program below is designed to send a sample query to your local Llama server and verify that it responds correctly.
The Python Program
Here’s the complete Python script to check your local Llama installation:
import requests
# Define the API endpoint of the local server
api_url = "http://localhost:11434/api/chat"
# Define the payload with the model name specified
payload = {
"model": "llama3.1",
"messages": [
{"role": "user", "content": "What is the capital city of Canada?"}
],
"stream": False
}
headers = {
"Content-Type": "application/json"
}
# Make the request to the local server
response = requests.post(api_url, json=payload, headers=headers)
# Check if the request was successful
if response.status_code == 200:
# Print the response from the server
print(response.json())
else:
# Print an error message
print(f"Error: {response.status_code}, Message: {response.text}")How It Works
API Endpoint:The api_url variable defines the endpoint of your locally running Llama server. In this example, it assumes the server is running on http://localhost:11434. Make sure the port matches your local setup.
Payload:The payload specifies the model (llama3.1) and the message we want to send. In this case, the query is simple: “What is the capital city of Canada?”
Headers:The script sets the Content-Type to application/json, ensuring the server processes the request correctly.
Response Handling:The program sends a POST request to the server using the requests library. If the response status code is 200 (success), it prints the response JSON. Otherwise, it displays an error message with the status code and error details.
Running the Script
Save the script as test_llama.py (or any name you prefer).
Ensure your Llama server is running locally.
Run the script using Python:python test_llama.pyExpected Output
If everything is set up correctly, the program will query your Llama server and return the response. For example:{
"choices": [
{"message": {"content": "The capital city of Canada is Ottawa."}}
]
}
If you encounter any errors, the script will output the status code and error message to help you troubleshoot.
Key Takeaways
This program is a quick way to ensure your local Llama server is up and running.
Modify the payload to test different queries or models as needed.
Running Llama locally empowers you to explore AI capabilities while maintaining control over your data.
Feel free to customize this script to suit your specific use cases or integrate it into larger projects.
Conclusion
Testing your local Llama installation is a crucial step before diving into more advanced projects. With this simple Python program, you can confirm that your setup is working and ready to handle queries.
Happy coding, and welcome to the world of local AI models! If you have any questions or run into issues, drop a comment below or get in touch via my DevRazor contact page. Let’s make the most of this exciting technology together!
Comments