I tried out 'Gerbil,' an AI app that can run chat and image generation locally.
' Gerbil ' is a PC app that can run large-scale language models and image generation models locally. It looked interesting because it can be used as a chat AI and generate high-quality images using FLUX, so I decided to give it a try.
GitHub - lone-cloud/gerbil: A desktop app for running Large Language Models locally.
Gerbil is an app that can be used on Windows, macOS, and Linux. This time, I'll try running it on Windows. First, go to the download page and download the latest installer. This time, I downloaded 'Gerbil-Setup-1.8.5.exe.'
Once the installer has finished downloading, double-click it to run it.
If a security warning appears, click 'More Information.'
Click 'Run'.
Once the installer starts, click 'Next'.
Click 'Install'.
Click “Finish”.
Then, Gerbil will start, so click 'Get Started'.
Next, you will be asked to select the PC environment you want to run the program on. In this case, I want to run it on a PC equipped with NVIDIA GeForce RTX 3060 (12GB), so I clicked 'Download' to the right of 'Standard.'
Please wait a while as the necessary files will begin to download.
When the following screen appears, click 'LAUNCH'.
The download of the large-scale language model will begin. If you leave the default settings, '
Once the download is complete, the chat UI selection screen will appear. In this example, I selected 'Corpo UI,' which is similar to chat AI services such as ChatGPT, and clicked 'Set UI.'
This is the Gerbil chat screen.
Enter text in the input field at the bottom of the screen and click the send button.
The response was output.
The GPU usage while generating the response looks like this. You can see that the AI is running locally instead of on the cloud.
Gerbil also has an image generation function, so let's try it out. First, click the mode switch button at the top of the screen and then click 'Eject.'
When the confirmation screen appears, click 'Eject'.
When the model settings screen appears, click “Image Generation” and then click the select button in the “Model Preset” field to select a preset. In this example, I chose “FLUX.1”.
The download URL for the required model will be automatically entered, so just click 'LAUNCH'.
Wait a few moments for the model to finish downloading.
Once the download is complete, the original chat screen will be displayed. To switch to image generation mode, click the mode switch button and then click 'Stable UI.'
The image generation screen is displayed.
Scroll down the screen to see the prompt input field, step count, and other adjustment bars.
Enter a description of the image you want to generate in the 'Prompt' field and click 'Generate 1 image'.
After a while, the generated results will be displayed. On a PC equipped with an NVIDIA GeForce RTX 3060 (12GB), it took about 11 minutes to generate one 512x512 pixel image.
Looking at the task manager, we can see that there is not enough VRAM and that it is leaking into system memory. It seems that a large amount of VRAM is required to generate images smoothly with FLUX.1.
Click 'Images' to save the generated image.
A list of the images you have created so far will be displayed, so press and hold any image.
When a selection circle appears in the upper right corner of each image, click the image you want to save and click 'Download' to save the ZIP file containing the images.
Gerbil is developed based on ' koboldcpp ', which was forked from ' llama.cpp ', and there are plans to migrate it to a llama.cpp base in the future.
Related Posts:
