Giving Jarvis a Voice 🎙️

Three pieces, one loop

A voice assistant is your existing chat loop with two converters bolted on either end:

  🎤 you speak
      |
      v
  [ Speech-to-Text ]   turns audio into text        (STT, "transcription")
      |
      v
  [ your chat loop ]   the Jarvis you already built  (Lessons 55-58)
      |
      v
  [ Text-to-Speech ]   turns text into audio         (TTS)
      |
      v
  🔊 Jarvis speaks

The middle is done. This lesson is about the two ends, and about the honest cost of doing it well.

Speech to text

You record audio (or stream it live) and send it to a transcription model, which returns text. The dominant open model is OpenAI's Whisper, which you can run through a cloud API or, notably, entirely on your own machine.

# Cloud transcription: send an audio file, get text back.
# (Shape shown; the exact SDK depends on your provider.)

def transcribe(audio_path):
    """Turn a recorded audio file into text."""
    from openai import OpenAI      # pip install openai
    client = OpenAI()
    with open(audio_path, "rb") as audio:
        result = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio,
        )
    return result.text


# text = transcribe("question.wav")
# then feed `text` into your chat loop exactly as if the user had typed it
print("Record audio -> transcribe -> feed the text into the chat loop.")

Record audio -> transcribe -> feed the text into the chat loop.

# Local transcription: nothing leaves your machine. Private and free to run.

def transcribe_locally(audio_path):
    """Transcribe with a Whisper model running on your own hardware."""
    import whisper          # pip install openai-whisper
    model = whisper.load_model("base")      # tiny/base/small/medium/large
    result = model.transcribe(audio_path)
    return result["text"]


# Slower on a laptop, but your voice never leaves the room.
print("Local Whisper: private, free per-use, needs a decent machine.")

Local Whisper: private, free per-use, needs a decent machine.

Text to speech

The reverse: text in, audio out. Quality ranges from the flat robotic voice built into your operating system to cloud voices so natural they are unsettling.

# Cloud TTS: high quality, costs per character, needs the network.

def speak_cloud(text, out_path="reply.mp3"):
    """Generate natural-sounding speech from text."""
    from openai import OpenAI
    client = OpenAI()
    audio = client.audio.speech.create(
        model="tts-1",
        voice="alloy",
        input=text,
    )
    audio.stream_to_file(out_path)
    return out_path


print("Cloud TTS: natural voices, per-character cost, sends your text out.")

# Local/offline TTS: robotic but private, free, and works with no internet.

def speak_locally(text):
    """Speak using a fully offline engine."""
    import pyttsx3          # pip install pyttsx3
    engine = pyttsx3.init()
    engine.say(text)
    engine.runAndWait()


# The voice is dated, but nothing is sent anywhere and there is no per-use cost.
print("Local TTS: robotic, private, free, offline.")

Local TTS: robotic, private, free, offline.

The full voice loop

# The assembled shape. Each converter is a swappable function.

def voice_assistant(transcribe, chat_reply, speak):
    """A voice loop: listen, think, speak, repeat. Converters are injected."""
    messages = []
    while True:
        audio_path = record_until_silence()          # capture the user speaking
        user_text = transcribe(audio_path)
        print(f"You said: {user_text}")

        if "goodbye" in user_text.lower():
            speak("Goodbye.")
            break

        messages.append({"role": "user", "content": user_text})
        reply = chat_reply(messages)                 # your Lesson 55 loop
        messages.append({"role": "assistant", "content": reply})

        print(f"Jarvis: {reply}")
        speak(reply)


def record_until_silence():
    """Capture microphone audio until the speaker pauses. Uses a mic library."""
    ...      # sounddevice / pyaudio; returns a path to the recorded clip


print("The loop is Lesson 55 with a microphone on the front and a speaker on the back.")

Notice the design: transcribe, chat_reply and speak are passed in, so you can swap cloud for local without touching the loop. That is the dependency-injection idea from Lesson 37, and it is what lets you choose per-piece between quality and privacy.

The trade-offs, honestly

The privacy question you must answer

Mark lesson complete +100 XP