Live Transcripts

Quartr API includes live transcribed transcripts during live calls. These transcripts are automatically generated, providing a textual representation of the audio content in real-time.

Understanding the Live Transcript FileCopied!

An example stream can be found here.

This file is initiated as soon as the live audio begins, although there might be a delay before the first spoken words are transcribed. Each line of the JSONL file represents a segment of the transcript, detailing the text spoken, its start and end times, and a unique paragraph identifier. The timestamps in the liveTranscriptJsonUrl match the audio of the live audio stream found in the liveUrl in the liveState of an event.

For instance:

{"s":581.4,"e":581.64,"p":"18","t":"that","S":"0"}

{"s":582.84,"e":583.34,"p":"18","t":"considered","S":"0"}

Explanation of the JSONL FormatCopied!

The live transcript is a JSON Lines (.jsonl) file, where each new line is a JSON object containing a fragment of the transcript. These fragments come with various attributes, such as start (s) and end (e) times, transcript text (t), and a paragraph identifier (p) as well as a speaker identifier (S). Additionally, some fragments may include an optional text (ot) field, which provides a substitute for indiscernible words. Words with low confidence are marked with [Indiscernible], and in some applications, these may not be displayed at all.

How to Use the Live TranscriptCopied!

To effectively use the live transcript, clients should poll the .jsonl file periodically—every 1-4 seconds is recommended—to retrieve updates without reloading the entire file, which would be inefficient. This can be accomplished by using the Range header in the HTTP request, setting the range to start from the byte after the last received content. This method ensures that only new data is fetched, keeping the process lightweight. Once the type:end line appears in the transcript, polling can cease, indicating the live event's conclusion.

For the technical implementation of range requests, refer to the Mozilla documentation on single-part range requests, which provides detailed guidance on the process and/or see the following example.

Understanding Special Transcript EventsCopied!

In addition to the live transcription data, there are specific events and messages that occur during the live call. These are used to communicate the state of the live transcript process and provide contextual information. Below is an overview of these events, their format, and how they can be utilized in your application.

Start Event

The start event marks the beginning of the live transcript. Until this event appears in the file, no transcript data is available. The event also indicates the version of the transcript file format, which is important for understanding which features are supported. New features may be introduced in future versions, but updates will remain backward compatible. Clients should ignore any unrecognized items to ensure compatibility with future versions.

Example:

{"type": "start", "file_format_version": "1.3"}

Interruption Event

The interruption event occurs if there is an issue with the transcription service during the live call. This event can be used to inform the user that transcription is temporarily unavailable, potentially displaying a message like "Reconnecting." The system will attempt to recover, and multiple interruption events may be recorded. If restarting is set to false, it means the transcription service has failed to recover. This is a rare occurrence, most often, the restart will be followed by a resumption of normal messages.

Example of a temporary interruption:

{"type": "interruption", "time": 0.6, "restarting": true}
{"t": "welcome", "s": 0.7, "e": 1.1, "p": "0", "S": "0"}

Example of a failure to restart:

{"type": "interruption", "time": 0.6, "restarting": true}
{"type": "interruption", "time": 0.7, "restarting": true}
{"type": "interruption", "time": 0.8, "restarting": true}
{"type": "interruption", "time": 0.9, "restarting": false}

Section Event

The section event marks the start of a new segment in the call. For example, in earnings calls, the system can detect the transition between the "Prepared Remarks" and the "Q&A" section. This is useful for clients who want to indicate specific portions of the call, such as when the Q&A session begins.

Example of an indication that Q&A started:

{"type": "section", "name": "predicted-qna", "s": 1.09}

End Event

The end event signals the conclusion of the live call and the transcript file. When this event is encountered, the client can safely stop polling the transcript for updates.

Example:

{"type": "end"}

Low Confidence Transcriptions

For transcript text with low confidence, the system may mark certain words as [indiscernible]. In such cases, the original transcription is stored in the ot (original text) field. This helps the client display the best possible transcription while noting where the system was uncertain.

Example:

{"t": "[indiscernible]", "s": 3, "e": 3.01, "p": "0", "c": 0.41, "ot": "first"}

Code ExampleCopied!

In the next section, you will find a code example demonstrating how to continuously stream an ongoing live transcript file, ensuring your live transcripts are up-to-date and accurately synchronized with the corresponding audio streams. See repository for a full example.

function startStreaming(liveJsonTranscriptUrl) {
  let range = 0;
  const interval = 2000

  const refetchStream = setInterval(() => {
    // test loop: https://ds.quartr.com/loop/live_transcript.jsonl
    axios({
      method: "get",
      url: liveJsonTranscriptUrl,
      responseType: "stream",
      headers: {
        "Content-Type": "application/json",
        range: `bytes=${range}-`,
      }
    }).then(function (response) {
      const rangeHeader = response.headers["content-range"];

      range = rangeHeader.split("/")[0].split("-")[1];

      const lineReader = readline.createInterface({
        input: response.data,
        crlfDelay: Infinity
      });

      lineReader.on("line", (line) => {
        const record = JSON.parse(line);
        handleRecord(record);
      });
    }).catch(function (error) {
      console.error("Error streaming transcript:", error);
      clearInterval(refetchStream)
    });
  }, interval)
}