Get audio effects detection insights

Audio effects detection detects acoustic events and classifies them into categories like laughter, crowd reactions, alarms, or sirens.

Audio effects use cases

• Improve accessibility by offering more context for a hearing- impaired audience by transcription of nonspeech effects.
• Improving efficiency when creating raw data for content creators. Important moments in promos and trailers such as laughter, crowd reactions, gunshots, or explosions can be identified, for example, in Media and Entertainment.
• Detect and classify gunshots, explosions, and glass shattering in a smart-city system or in other public environments that include cameras and microphones.

Supported audio categories

Audio effects detection can detect and classify effects into standard and advanced categories. For more information, see pricing.

The following table shows which categories are supported depending on Preset Name (Audio Only / Video + Audio vs. Advance Audio / Advance Video + Audio). When you're using the Advanced indexing, categories appear in the Insights pane of the website.

View the insight JSON with the web portal

After you upload and index a video, download insights in JSON format from the web portal.

1. Select the Library tab.
2. Select the media you want.
3. Select Download, and then select Insights (JSON). The JSON file opens in a new browser tab.
4. Find the key pair described in the example response.

Here's an example showing the audio effects detection.

Use the API

1. Use a Get Video Index request. Pass &includeSummarizedInsights=false.
2. Find the key pairs described in the example response.

Example response

    "audioEffects": [
      {
        "id": 1,
        "type": "Silence",
        "instances": [
          {
            "confidence": 0,
            "adjustedStart": "0:01:46.243",
            "adjustedEnd": "0:01:50.434",
            "start": "0:01:46.243",
            "end": "0:01:50.434"
          }
        ]
      },
      {
        "id": 2,
        "type": "Speech",
        "instances": [
          {
            "confidence": 0,
            "adjustedStart": "0:00:00",
            "adjustedEnd": "0:01:43.06",
            "start": "0:00:00",
            "end": "0:01:43.06"
          }
        ]
      }
    ]

Sample code

See all samples for VI

Closed captions

Audio effects in closed caption files appear as square brackets:

Add audio effects to closed caption files

To include audio effects in closed captions files, you can use the API or the web portal.

API

You can add audio effects to closed captions files with the Get video captions request and by choosing true for the includeAudioEffects parameter.

Web portal

You can also use the web portal by selecting Download -> Closed Captions -> Include Audio Effects.

Related content

• Get keywords extraction insights