GitHub

Project Description: RTranslator

What is the project about?

RTranslator is an (almost) open-source, free, and offline real-time translation app for Android devices. It enables users to have conversations with others who speak different languages, simulating simultaneous translation.

What problem does it solve?

It breaks down language barriers in real-time conversations, allowing people who speak different languages to communicate easily and naturally, even without an internet connection. It addresses the limitations of traditional translation methods that are often slow, require internet connectivity, or lack privacy.

What are the features of the project?

  • Conversation Mode: Connects two or more Android devices via Bluetooth for real-time, two-way translation. Supports Bluetooth headsets.
  • WalkieTalkie Mode: For quick, turn-based conversations between two people using a single phone. Translates between two selected languages automatically.
  • Text Translation Mode: A standard text translator for written communication.
  • Offline Functionality: Works entirely offline, ensuring privacy and usability in areas with no internet access.
  • Background Operation: Can translate even when the app is in the background or the phone is on standby (though some phones may limit background processing).
  • Multi-person conversations: Each user can connect to multiple phones.
  • Low-quality language support: Option to enable languages with lower translation and speech recognition quality.
  • Customizable Text-to-Speech (TTS): Uses the phone's system TTS, allowing users to choose their preferred TTS engine.

What are the technologies used in the project?

  • AI Models:
    • Meta's NLLB (No Language Left Behind) for text translation (specifically, NLLB-Distilled-600M with KV cache).
    • OpenAI's Whisper for speech recognition (specifically, Whisper-Small-244M with KV cache).
  • Libraries:
    • BluetoothCommunicator (open-source): Bluetooth LE communication.
    • GalleryImageSelector (open-source): Profile image selection.
    • OnnxRuntime (open-source): AI model accelerator.
    • SentencePiece (open-source): Text tokenization for NLLB.
    • Ml Kit (closed-source): Language identification in WalkieTalkie mode.
  • Other:
    • Android's system Text-to-Speech (TTS) engine.
    • ONNX format and int8 quantization for model optimization.

What are the benefits of the project?

  • Real-time Communication: Facilitates near-simultaneous translation for natural conversations.
  • Offline Functionality: Usable anywhere, regardless of internet connectivity.
  • Privacy: No data collection; all processing happens on the device.
  • Free and (Almost) Open-Source: Accessible to everyone.
  • Optimized Performance: AI models are optimized for mobile devices, reducing RAM usage and execution time.
  • Customizable: Users can select their preferred TTS engine.

What are the use cases of the project?

  • Travel: Communicating with locals in foreign countries.
  • Business: International meetings and negotiations.
  • Education: Language learning and cross-cultural exchange.
  • Personal Relationships: Connecting with friends and family who speak different languages.
  • Emergency Situations: Communicating with first responders or others in multilingual environments.
  • Any situation where real-time translation between two or more people is needed.
RTranslator screenshot