An Introduction to Basic Tools and Concepts of AI
I will give y ou a definition of a proud man: he is a man who has neither vanity nor wisdom one filled with hatreds cannot be va in, neither can he be wise.
— John Keats
Hello, here is Wesley, Today’s article is about Basic Tools and Concepts of AI . Without further ado, let’s get started.💪
Introduction
I’m used to trying out new things first and then learning more about them afterwards. This applies to AI as well. Although AI has become a part of my daily life, I didn’t even know what some basic AI concepts meant. To me, this was too passive, and I felt like I should take the initiative to learn about it. After all, AI is here to stay.
This article will first introduce how I use AI in my daily life, and then follow up with some basic AI concepts that can help us keep up with the latest developments. These concepts are mostly based on two websites:
If you think these AI basic concepts are boring, you can ignore them (although I don’t recommend it).
Integrating AI into Life
Before that, let me introduce my computer configuration:
- MacOS + M1 Pro 32G
I’ve also categorized this section, so you can choose the scenario you want to try out and use.
Write: Obsidian
Use GitHub - logancyang/obsidian-copilot: A ChatGPT Copilot in Obsidian.
Usage scenarios and features:
- Translate text into English, without needing to copy it to an AI website.
- Expand content
- Supports localized LLMs like Ollama
- ….
Example: Using Emojify selection
More:
Code: VSCode
Use the plugin: GitHub - continuedev/continue: ⏩.
Usage scenarios and features:
- Explain code
- Code completion
- Supports localized LLMs like Ollama
- Jetbrain IDEs can also be used
- …..
More:
Browse: Chrome
ChatGPTBox
Usage scenarios:
- Provide answers for Chrome searches
- Summarize web pages and videos
Example with Gemini:
- Search question
- Video summary
Page Assist - A Web UI for Local AI Models
Usage scenarios:
- Supports localized LLMs like Ollama
- Assists with article-related scenarios, such as asking questions
Mail and Notes
Reference: System-wide text summarization using Ollama and AppleScript for setup
PS: When you first use it, you can run the following command to install the model. (Note: How to install Ollama will be introduced later.)
1 | ollama run mistral |
Usage scenarios and features:
- Your notes, email
- Uses localized LLMs like Ollama
Mobile (iOS) Browse
I occasionally use Arc browser on my phone for searching, mainly utilizing two features:
- Summarize entire pages with a pinch: Get the gist, in a pinch. Pinch to Summarize, now available in Arc Search
- Browses for you: Arc Search for iPhone | The fastest way to search - YouTube
Model Selection
Non-Local Models
ChatGPT, Claude.ai, Bing, Gemini, Poe, OpenRouter, and more - you can try using them. Most online AI-related content is about introducing these models, so I won’t elaborate further.
Local Models
Ollama
GitHub - ollama/ollama: Get up and running with Llama 3, Mistral, Gemma, and other large language models.
Ollama is an open-source large language model service tool that provides features for downloading, managing, and reasoning with models, as well as serving APIs for other programs to call. It has a thriving ecosystem.
The Ollama model files are located at: ～/.ollama/models, which can be discovered using the command:
1 | ollama show starcoder2:3b --modelfile |
Other installation and usage commands can be viewed, such as:
1 | ollama -h |
To use Ollama to download the first model, it’s likely Llama3now, which is introduced here: Introducing Meta Llama 3: The most capable openly available LLM to date
LM Studio
LM Studio - Discover, download, and run local LLMs
Ollama is great, but for ordinary users, it’s still too complicated to use. LM Studio is a tool with a built-in UI that’s easy to get started with, although it’s not open-source and currently free to use.
LM Studio supports searching for models, downloading, managing, and running them in one place, not just Chat, but also multiple model sessions and Local LLM Server.
The LLM model directory is located at: ~/.cache/lm-studio/models/.
I’ve tried it out and found it to be quite good, but my computer takes a bit of effort to run it, so I don’t use it often.
AI Basic Concepts
Firstly, thank you for reading this far. As a programmer, my AI application scenarios are basically these tools, which have significantly improved my productivity, and I hope they will help you too.
If you’re using these AI tools for the first time, I’m sure you’ll have one question: What model should I install?
This requires you to understand related concepts and make judgments based on your own needs.
LLM & Prompting
Large language model - Wikipedia, everyone should be familiar with this concept, so I won’t elaborate further. The models we can run locally are mainly Llama provided by Meta, including Llama2 and Llama3. Of course, there are also challengers in this field, such as Mistral AI | Frontier AI in your hands
Prompt engineering - Wikipedia, prompt engineering is a promising technique that has the potential to revolutionize the way we interact with LLMs.
Promting | How-to guides provides a more in-depth introduction. I recommend reading it and trying it out based on your own needs.
Parameters
Term origin: available in both 8B and 70B parameter sizes
“B” refers to billions of parameters, which are typically considered to be a measure of model quality, with more parameters generally leading to better performance. For example, Llama3 provides models in 8B and 70B sizes:
Quantization
Term origin: quantization Q4_0
If a full-size model uses 2 bytes of storage per parameter (typically referring to 16-bit floating-point numbers or FP16 representation), this means that a 3B model’s actual size is 6GB. If we were to quantize the above 3B parameter model from FP16 (2 bytes) to 8-bit (1 byte), storage would further reduce to 3GB.
Here, 8-bit refers to Q8, and the original text’s Q4 likely refers to quantization to 4-bit. Using this example, with Q4 technology, storage would further reduce to 1.5GB.
Fine-Tuning
Term origin: Llama 3 instruction-tuned models are fine-tuned and optimized for dialogue/chat use cases
Fine-tuning is a technique in deep learning and machine learning that involves further training on top of pre-trained models to adapt to specific tasks or datasets. This process typically requires fewer computational resources and time, as the pre-trained model has already learned a large number of general features, and fine-tuning only needs to make minor adjustments.
Context
Term origin: 200,000 context
It refers to the range of input information that a model considers when generating responses or making predictions. Since LLM is stateless and does not retain historical dialogue, you need to input all historical dialogue to make it understand and provide correct responses. Specifically, “context” can be understood as the text segment or information range that a model can see when processing a task.
For example, language models have a fixed “context window” size, which determines the maximum input length the model can handle. For GPT-3-like models, this context window may be 2048 tokens or more, meaning that the model can only consider tokens within the window when generating responses.
Others: temperature, top_p, top_k…
Reference: Mistral: Mistral 7B Instruct – Recommended Parameters | OpenRouter
Summary
First, I’d like to ask you a question: do you have an initial impression of the concepts above? If not, don’t worry – you can now use AI to learn on your own. Believe me, once you start using it, you’ll understand it even deeper.
This article mainly does some popularization and information summarization work in two aspects:
- I used some out-of-the-box AI tools
- Some basic AI concepts
I hope this can help you choose the right model and let AI serve you better. If it can also spark your interest in AI and keep up with the latest AI trends, that’s even better.
By the way, did you notice that most of the scenarios mentioned above can be used with your own local models, such as Ollama? This can help us protect our privacy – isn’t that great!
I think this will be a future development trend. In the near future, there will definitely be a customized AI model for you, and let’s look forward to it together!
More
- What are the strengths of different AI models from various manufacturers?
- Follow AI news and learn about the current state of AI development
- Use API interfaces provided by AI manufacturers to develop code
- ……
Have fun!!!
References
System-wide text summarization using Ollama and AppleScript
More Series Articles about Stay Ahead of The Curve：
https://wesley-wei.medium.com/list/stay-ahead-of-the-curve-45566b07b4a2
And I’m Wesley, delighted to share knowledge from the world of programming.
Don’t forget to follow me for more informative content, or feel free to share this with others who may also find it beneficial. it would be a great help to me.
See you in the next article. 👋
中文文章: https://programmerscaree r.com/zh-cn/use-llm-local/
Author: Wesley Wei – Twitter Wesley Wei – Medium
Note: Originally written at https://programmerscareer.com/use-llm-local/ at 2024-06-28 01:27. If you choose to repost or use this article, please cite the original source.
Comments