NativePrompt

Low Latency, Offline, In-place Prompt Translator

NativePrompt is a lightweight, high-performance utility designed to instantly translate text from your native language into English using a simple hotkey. It operates completely offline, ensuring your data remains private and providing near-instantaneous translations.

The primary goal of NativePrompt is to reduce the cognitive load associated with writing prompts for Large Language Models (LLMs). Since most state-of-the-art LLMs perform best with English prompts, this tool allows you to think and write in the language you're most comfortable with, then seamlessly translate your prompt into high-quality English before sending it to an LLM.

It's particularly useful when working with local LLM interfaces like LM-Studio, Silly Tavern, and others.

Key Features

• Low Latency: Translations are practically instantaneous.
• Offline First: After the initial model download, no internet connection is required. Your prompts stay on your machine.
• Low Resource Usage: The translation model occupies only about 1GB of VRAM.
• High-Quality Models: Powered by the robust Helsinki-NLP/Opus-MT models, known for their excellent translation quality.
• Wide Language Support: Supports over 99+ languages (one source language at a time, based on the model you download during setup).
• Seamless One-Press Workflow: Press a hotkey in any text field to instantly translate and replace the entire content. No manual selection required.

Important Limitations

1. No Spell/Grammar Correction: The translator will not fix typos or grammatical errors in your source text. The quality of the translation depends on the quality of your original prompt. Ensure your text is written correctly before translating.
2. Not Suitable for Programming/Code: The model will attempt to translate variable names, function names, and syntax, even if they are already in English. This will break your code. Do not use this tool on source code.

Installation

Setting up NativePrompt is straightforward.

Prerequisites

• A Windows operating system.
• Python 3.0+ installed and added to your system's PATH.
• An NVIDIA GPU with CUDA installed (the script is configured for CUDA 12.4).

Setup Steps

1. Download or clone this repository to your local machine.
2. Double-click the setup.bat script.

This script will automatically:

• Create a local Python virtual environment in a folder named venv.
• Activate the environment.
• Install all the necessary packages, including PyTorch, Transformers, and other dependencies.

The setup process may take a few minutes, especially the PyTorch installation.

How to Use

1. Launch the Translator:

   • Double-click the run.bat file.
   • This will activate the correct environment and start the NativePrompt.py script in a new console window.

2. First-Time Model Setup:

   • On the very first run, the script will prompt you to enter the source language code for the language you want to translate from (e.g., de for German, fr for French, it for Italian, ru for Russian).
   • The script will then download and save the corresponding translation model locally. This only happens once.

3. Translate Your Text:

   • Once the model is loaded, you will see a message: Ready to translate...
   • Go to any application (a text editor, a web browser, LM-Studio, etc.).
   • Place your cursor inside the text field you want to translate.
   • Press the hotkey: F9.
   • NativePrompt will automatically select all the text in the field, translate it, and instantly replace the original with its English version.

   💻 Note for Laptop Users: On many laptops, the top row keys control system functions (like brightness and volume) by default. To use F9, you may need to hold the Fn key, making the combination Fn + F9.

4. Exiting the Program:

   • To stop the translator, simply close the console window where NativePrompt is running, or press Esc in that window.

Changing the Hotkey

The default hotkey is F9. While this key is reliable, it might be used by specific applications (e.g., for debugging in some code editors). If you find F9 inconvenient, you can easily change it.

1. Open NativePrompt.py in a text editor.
2. Find this line near the top of the file:

   HOTKEY = 'F9'

3. Change it to a combination that you prefer, for example:

   HOTKEY = 'ctrl+alt+t'

4. Save the file. The new hotkey will be active the next time you run run.bat.

---

Credits

This project relies heavily on the fantastic work done by the University of Helsinki's NLP group.

• Models: Helsinki-NLP for the high-quality Opus-MT translation models.
• Framework: Hugging Face Transformers for providing easy access to these models.