Last updated:

Interactive Markers and Transcript for WaveSurfer-WP Premium

I originally wrote this system for WaveSurfer-WP Premium, but since the writing of this article, I succeed to port this to YouTube embeded video player and SoundCloud. It would need a bit more work for a WordPress integration but the demo are encouraging. Let me know in comments if you are interested!

Introduction

For my WaveSurfer-WP audio player WordPress plugin, I wanted to have a marker system, which could allow to add clickable buttons on a webpage to set the player cursor at a dedicated timecode.

The main idea was to be able to mark certain chapters, for audio podcasts and audio stories, for example.

I wanted the system to be the more flexible possible, so I push the thing further and… you can now use this marker system for interactive transcripts!

On the following example, you will see various integration examples of this exciting feature.

The following examples used audio from the Sal Khan’s: Let’s teach for mastery — not test scores TED Conference.
TED.com doesn’t provide native ways to download subtitles, so they were downloaded thanks to the Ted 2 SRT website, and adjusted with the great SubtitleEdit open source software.

Basic Usage

I made an update of the script so I can manage more human readable format like 04:05.15 timecodes.

You can put the wavesurfer-marker HTML/CSS class on any element to create a marker. Style customization (design integration) is up to you.

Then you can add attributes to specify your markers info.

To make the following code snippets more readable, I will omit the data-id attribute, just like for pages with only one WaveSurfer-WP player.

Simple Markers

This is the most basic usage. You only need to add the wavesurfer-marker class and a data-start attribute.

DemoHTMLCSSExplanations
  1. 0:17: Introduction
  2. 2:28: Core
  3. 3:48: Home Building Metaphor

In this case, the class is adding to a simple span surrounding a timecode, but you can put it on a button, just like in this post featured image.

Interactive Transcripts and Lyrics

Concept

Having the full transcript of an audio track on a web page is very nice for several reasons:

  • SEO friendly: search engine will be able to know what is said in your audio
  • Users can read the text as they would read an article
  • Users can perform quick search on the web page to find a particular word in the text

On Ted.com, for the conferences, and on Deezer, for lyrics, this transcript are interactive: you can click on certain sentences, or group of words, and it will put the audio play cursor at the desired place. Very cool!

I wanted to have such system for WaveSurfer-WP, and after few tests I realize that my marker system could do the job!

Subtitles Conversion

Most transcripts with time informations are created and shares as subtitles. As SRT subtitles are the most common subtitles format, I coded a SRT to WaveSurfer-WP Markers converter. This tool allow users to convert their SRT into HTML paragraphs with WaveSurfer-WP Markers. They can then put the results in their WordPress article.

It features several options to customize the output. As it is shared on CodePen, you can fork it to put the customization even further if needed.

Demo

Here is the sound and an extract of its interactive transcript based on its SRT subtitles.

DemoHTMLCSSExplanations

I’m here today to talk about the two ideas that, at least based on my observations at Khan Academy, are kind of the core, or the key leverage points for learning.

And it’s the idea of mastery and the idea of mindset.

I saw this in the early days working with my cousins.

A lot of them were having trouble with math at first, because they had all of these gaps accumulated in their learning.

And because of that, at some point they got to an algebra class and they might have been a little bit shaky on some of the pre-algebra, and because of that, they thought they didn’t have the math gene.

Or they’d get to a calculus class, and they’d be a little bit shaky on the algebra.

I saw it in the early days when I was uploading some of those videos on YouTube, and I realized that people who were not my cousins were watching.

(Laughter)

And at first, those comments were just simple thank-yous.

I thought that was a pretty big deal.

A SRT converted in WaveSurfer markers thanks to the CodePen code snippet.

A simple CSS rule allows to see the current subtitles, from audio play position.

Subtitles

DemoHTMLCSSExplanations

Intro

I’m here today to talk about the two ideas that,

at least based on my observations at Khan Academy,

are kind of the core, or the key leverage points for learning.

And it’s the idea of mastery

and the idea of mindset.

I saw this in the early days working with my cousins.

A lot of them were having trouble with math at first,

because they had all of these gaps accumulated in their learning.

And because of that, at some point they got to an algebra class

and they might have been a little bit shaky on some of the pre-algebra,

and because of that, they thought they didn’t have the math gene.

Or they’d get to a calculus class,

and they’d be a little bit shaky on the algebra.

I saw it in the early days

when I was uploading some of those videos on YouTube,

and I realized that people who were not my cousins were watching.

(Laughter)

And at first, those comments were just simple thank-yous.

I thought that was a pretty big deal.

I don’t know how much time you all spend on YouTube.

Most of the comments are not “Thank you.”

(Laughter)

They’re a little edgier than that.

— Go Back in the audio, I don’t put the whole transcript, as it is only a demo —

The subtitles have been converted without merging.

A CSS rule allows to display only the current subtitles, based on play cursor position.

For subtitles display, you may consider adjusting subtitles timing, so that they end precisely at the next subtitles position to prevent text to flash.

Extra

If you convert every subtitles to its own paragraph, and center the text, you will have a lyrics type of transcripts display, ala Deezer.

And if you divide timing at every syllabus, you could even make a Karaoke system!

Also, if you need to reinitialize markers for a transcript (for eg, if you have an dropdown which loaded another transcript in AJAX,) you can use the wavesurfer-markers-init JavaScript event.

Conclusion

I had to discover a lot of things to make this possible (it was the first time I use the FileReader API), but it was quite fun to learn.

A flexible system for markers and interactive transcripts open a whole new world of possibilities. I hope WaveSurfer-WP users will take advantages of this great features, added to WaveSurfer-WP Preimum Add-On ! 😛