Put GPT on the Raspberry Pi

• Speech-to-text and text-to-speech with offline wake-word detection using Azure Speech Services.
• Complete a single-round question-answering text using OpenAI ChatGPT 3.5 Turbo.
• Multi-turn and context-aware chat using OpenAI ChatGPT 3.5 Turbo Chat.
• Use Azure Computer Vision services for image analysis and captioning.
• Raspberry Pi Camera Module v3 with image capture and sharing capabilities.
• The new painting mode uses stable diffusion to paint pictures based on your spoken descriptions.
• New DALL-E image from ChatGPT 3.5 description mode, which will create an image from a description generated by ChatGPT.

Large Language Model

pip3 install llama-cpp-pythonpip3 install huggingface-hub sentence-transformers langchain

Before using LLM, you need to download it. Use the huggingface-cli tool:

huggingface-cli download TheBloke/Llama-2-7b-Chat-GGUF llama-2-7b-chat.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks FalseORhuggingface-cli download TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False

The Llama-2–7b-Chat-GGUF and TinyLlama-1–1B-Chat-v1-0-GGUF models were used. Smaller models run faster, but larger models may provide better results.

from langchain.llms import LlamaCppfrom langchain.callbacks.manager import CallbackManagerfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerfrom langchain.prompts import PromptTemplatefrom langchain.schema.output_parser import StrOutputParser
llm: Optional[LlamaCpp] = Nonecallback_manager: Any = None
model_file = "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" # OR "llama-2-7b-chat.Q4_K_M.gguf"template_tiny = """<|system|>You are a smart mini computer named Raspberry Pi.Write a short but funny answer.</s><|user|>{question}</s><|assistant|>"""template_llama = """<s>[INST] <<SYS>>You are a smart mini computer named Raspberry Pi.Write a short but funny answer.</SYS>>{question} [/INST]"""template = template_tiny
def llm_init():""" Load large language model """global llm, callback_manager
callback_manager = CallbackManager([StreamingCustomCallbackHandler()])llm = LlamaCpp(model_path=model_file,temperature=0.1,n_gpu_layers=0,n_batch=256,callback_manager=callback_manager,verbose=True,)
def llm_start(question: str):""" Ask LLM a question """global llm, template
prompt = PromptTemplate(template=template, input_variables=["question"])chain = prompt | llm | StrOutputParser()chain.invoke({"question": question}, config={})

class StreamingCustomCallbackHandler(StreamingStdOutCallbackHandler):""" Callback handler for LLM streaming """

def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) -> None:""" Run when LLM starts running """print("<LLM Started>")

def on_llm_end(self, response: Any, **kwargs: Any) -> None:""" Run when LLM ends running """print("<LLM Ended>")

def on_llm_new_token(self, token: str, **kwargs: Any) -> None:""" Run on new LLM token. Only available when streaming is enabled """print(f"{token}", end="")add_display_tokens(token)

test

if __name__ == "__main__":add_display_line("Init automatic speech recogntion...")asr_init()

add_display_line("Init LLaMA GPT...")llm_init()

while True:# Q-A loop:add_display_line("Start speaking")add_display_line("")question = transcribe_mic(chunk_length_s=5.0)if len(question) > 0:add_display_tokens(f"> {question}")add_display_line("")

llm_start(question)

AI Polaroid Photo Implementation Using GPT+Raspberry Pi

Make a watch that supports ChatGPT