//
you're reading...
Audio

Extracting Audio from Pictures

Three years ago, a survey team identified a staggering 569,148 time-based media objects on the Bloomington campus of Indiana University—that’s over half a million sound recordings, video recordings, and reels of film. The earliest items documented in the survey report date back to 1893.

However, that report doesn’t mention what might be considered IU Bloomington’s oldest time-based media of all. That’s not because we didn’t do a thorough job. Rather, it’s because the items in question don’t look or behave much like the media we were surveying—discs, reels, cylinders, cassettes, and so forth. Instead, they’re pictures in books. And pictures in books seem well outside the scope of the Media Preservation Initiative.

But that doesn’t mean we can’t play them back—and some of them are pretty exciting.

"Der Handschue" as an image

A printed replica of the early gramophone disc “Der Handschuh”

Just a few months ago, I crossed the fourth floor of Wells Library from the elevators, located the volume with call number AP30 .U22 v.63 in the oversized shelving on the back wall, and opened it up to page 395 to reveal a circular image—on closer inspection, a nearly microscopic wavy line coiled into a tight spiral.

Close-up of "Der Handschue"

A closer inspection of the image of “Der Handschuh” reveals imprints of the original record’s grooves.

This isn’t just a pretty picture.  It’s a bona fide sound recording—a “record.”  In fact, it might arguably be the oldest “record” in the world that you can listen to today!

Let me clarify—I don’t mean it’s the world’s oldest sound recording.  But nowadays when people use the word “record” colloquially to refer to sound media, they typically mean the specific format that includes LPs, 45s, and 78s—that is, the kinds of grooved disc you’d play on a “record player.” Technically, these “records” are based not on the phonograph Thomas Edison unveiled in 1877, but on the gramophone invented by Emile Berliner in 1887.  The gramophone disc dominated the worldwide recording industry for much of the twentieth century and still has currency in the twenty-first, for instance in the art of turntabling.  The distinctive crackle of its surface noise is stamped in the popular imagination as the quintessential “old recording” sound.

So what are the oldest known “records” in this sense—that is, the oldest known gramophone recordings, as opposed to the oldest sound recordings in general?  The first commercially available gramophone discs were manufactured and released in Europe in the summer of 1890, and numerous examples are available for listening (here, for example).  In addition to these, a few experimental gramophone discs from 1887 and 1888 survive at the Smithsonian Institution and elsewhere, but attempts to play these haven’t been very successful, and no intelligible or identifiable content has been recovered from them to date.  Finally, some other very old gramophone recordings have come down to us only in the form of prints made on paper,like the one on the fourth floor of Wells Library.  This isn’t a unique situation.  Many important early motion pictures that didn’t survive in the form of actual films were nevertheless preserved as paper prints deposited for copyright registration purposes with the Library of Congress and later retransferred to film for projection and preservation.  Similarly, I’ve found that paper prints of “lost” gramophone recordings can be digitally converted back into playable, audible form.

Here’s how it works.  First, I take a high-resolution scan of the print and convert it from a spiral into a set of parallel lines through a polar-to-rectangular-coordinates transform.

The first step of converting an image to sound

A high-resolution scan of a printed record, converted from a spiral into a set of parallel lines through a polar-to-rectangular-coordinates transform.

Next, I “cut” the individual lines and “paste” them end-to-end to create several long, narrow strips. 

Cut-and-Pasted Images to Soundwaves

A waveform image from “Der Handschuh” being cleaned up for conversion to audio.

After repairing any breaks in the line, I use a “paintbucket” tool to create two separate bands of varying width—one with the area below the line filled in white, the other with the area above the line filled in white.  Next, I run these images through ImageToSound, a program that converts them into WAV files as though they were variable-area optical film sound tracks.  Finally, I combine the paired WAV into stereo files, stitch the successive pieces together, sum to mono, and voilà—we have sound!

Image to audio conversion, step 3

An original waveform image (top), a pair of processed images (middle), and a display of the final audio file (bottom).

By the time I stumbled across the example on the fourth floor of Wells Library, I’d already played back three paper prints of gramophone recordings in this way. One of them, a seemingly unique print preserved at the Library of Congress, revealed itself to be a German-language test recording Berliner had made in Hanover on 11 November 1889 to demonstrate his recording process for an out-of-town visitor named Louis Rosenthal, who—as Berliner explains—would be taking it home with him to use for photomechanical duplication experiments (these were quickly abandoned in favor of other methods). For the past couple years, this has remained the oldest available soundcaptured by and for Berliner’s gramophone, the modern turntable’s earliest direct ancestor. You can listen to it here.

But what about the newly discovered example in Wells Library? That one was published in February 1890 in Über Land und Meer, a German illustrated magazine, several months before the first gramophones were put on sale, so that curious readers could see what the new instrument’s “records” looked like. A caption identifies its content as the poem “Der Handschuh” by Friedrich Schiller. The author of the accompanying article recognized even then that the print contained enough information to reproduce the recorded speech, at least in theory; and he outlined a roundabout method readers of the time could have used to listen to it if they’d felt it was worth the trouble:

Let an engraved plate be made through heliotypy from the sound-pressing figure 3 and lain (center upon center) on a disc which is turned steadily at about fifty times per minute. Now, anyone who holds a bamboo stick about 15 centimeters long and ¾ centimeter thick, to the end of which a stout darning needle is attached such that 1 centimeter protrudes, at an angle between the teeth and presses the point of the needle with the finger softly into the groove, starting from the edge, hears clearly the “Handschuh” in the speech of the inventor, especially if both ears are stopped with cotton.

The “Handschuh” recording had been photographically reduced before publication to three quarters of its original size. Its original diameter, intaglio printing method, and 50 rpm speed match the distinctive specifications of the recording from 11 November 1889 described above. These points all suggest that the “Handschuh” was probably another sample disc Berliner had given to Rosenthal for use in his short-lived photomechanical duplication experiments. This time, however, Berliner doesn’t sound like he’s demonstrating the recording process for Rosenthal—instead, we simply hear him reciting the poem from start to finish.

Listen here:
Read along: “Der Handschuh” by Friedrich Schiller

After weighing the evidence, my colleague Stephan Puille and I conclude that Berliner had most likely demonstrated the recording process for Rosenthal on 11 November 1889 and then sent him home with the record they’d made together plus a few others Berliner had prepared previously.

If we’re right, “Der Handschuh” must be the older of the two recordings, making it the oldest gramophone recording available anywhere for listening today—the earliest audible progenitor of all the world’s vintage vinyl.

If we’re wrong, it’s still an extremely early gramophone recording, as well as the oldest known recording of a complete literary work in the German language—certainly nothing to sneeze at.

Either way, it’s right here at IU Bloomington.  Of course, other institutions also have holdings of Über Land und Meer, and some of them surely have the issue containing this recording.  But our copy is the one that got played back—and you don’t even need to hold a bamboo stick between your teeth to listen to it!

Some Other Early “Recordings” at IU
Here are a few other snippets of audio obtained from high-resolution scans of books in the IU Bloomington Libraries.

"A Course of Lectures on Natural Philosophy and the Mechanical Arts"

Audio from Thomas Young, “A Course of Lectures on Natural Philosophy and the Mechanical Arts”

Year: 1806

Lilly Library: Q113 .Y77 (two copies, one previously owned by Ian Fleming)
Thomas Young, A Course of Lectures on Natural Philosophy and the Mechanical Arts (London: Joseph Johnson, 1807), Volume 1, Plate XXV, Fig. 353.
Significance: Oldest known inscription of audio “waveforms,” not recorded automatically but drawn by hand.  (The book is dated 1807, but the engraving itself is dated 1806.)
Listen here:

"The Phonautograph"

Chas. A. Morey, “The Phonautograph”

Year: 1874
ALF (Geosciences): Q1 .A5 ser.3, v.8
Chas. A. Morey, “The Phonautograph,” American Journal of Science and Arts 108 (Aug. 1874), 130-31, on page 131.
Significance: Oldest known publication of sound recordings made in the United States.
Listen here:

“The Talking Phonograph"

“The Talking Phonograph”

Year: 1877
Wells Library (oversized): Q1 .S45 n.s.,v.37 1877
“The Talking Phonograph,” Scientific American 37 (December 22, 1877), 384-5, on page 384.
Significance: Print made from a plaster cast of a fragment cut from the sample tinfoil recording Thomas Edison used to demonstrate his phonograph for an audience outside his laboratory for the first time.  I’ve inserted silences to represent the missing content (which is a majority of it).  The direction of recording is anybody’s guess, so what you hear might be played backwards.
Listen here:


E. W. Blake, Jr., “A method of recording Articulate Vibrations by means of Photography"

E. W. Blake, Jr., “A method of recording Articulate Vibrations by means of Photography”

Year: 1878
ALF (Geosciences): Q1 .A5 ser.3,v.16
E. W. Blake, Jr., “A method of recording Articulate Vibrations by means of Photography,” American Journal of Science and Arts 116 (July 1878), 54-59, on page 57.
Significance: Oldest known publication of a recording of recognizable phrases in the English language (“Brown University”; “How do you do?”); also the oldest known publication of a photographic recording of airborne sounds.

Listen here:


// Patrick Feaster

Discussion

55 thoughts on “Extracting Audio from Pictures

  1. Reblogged this on Musical Traces.

    Posted by bigmikeydread | 06/21/2012, 2:44 pm
  2. this software used seems to no longer be on the web. Any one know how to get a copy or a program like it?

    Posted by dave | 12/01/2012, 3:54 am
  3. Absolutely incredible. The age of the recordings we are hearing is amazing in itself, but the fact they are played back from pictures is staggering … a salute to 19th century technology and a coup for today’s technology in one package. Hats off to Mr. Feaster !

    Posted by Joe Orbin | 01/18/2013, 8:14 pm
  4. Marvelous stuff! It’s remarkable that the published image of the disc (as opposed to images made for copyright purposes) actually reproduced the grooves accurately! I wonder whether your technique could be applied to photographs of discs? And I’m also curious as to whether you’ve tried this on photomechanical discs or even perhaps some of John Logie Baird’s “Phonovision” television recordings from the 1920’s — perhaps there are photos of some such discs, since lost, from which a signal could be recovered?

    Posted by Russell Potter | 04/05/2013, 10:41 am
    • That would be a fascinating way to test the accuracy of this method – take a photo of, say, a Beatles record, use this method, and hear how it sounds.

      Posted by Kevin | 04/05/2013, 12:37 pm
    • I think it also speaks volumes as to the quality of photography and reproduction. I know the pictures in most newspapers and magazines I read today are not printed at anywhere near high enough resolution to do this. Heck, even most color film photography may not have been, since the dyes it uses are usually at a lower resolution and dynamic range than the old timey silver granules of B&W photography.

      Posted by Brian | 04/06/2013, 12:25 pm
  5. Reblogged this on Kirsten's Amazing Resource Blog! and commented:
    This is pretty interesting. I have never thought of this before.

    Posted by cursetenblog | 04/05/2013, 4:21 pm
  6. This is the most amazing information I have happened upon for as long as I can remember. The concept of reproducing sound – recorded in the 19th century, no less – from prints of records in such old volumes is staggering. I am privileged that I got to hear a recording of speech from so long ago. Thank you.

    Posted by Richard | 04/05/2013, 4:50 pm
  7. The fact that this is absolutely exciting stuff put aside, technically, your method seems pretty laborious… Cant you get someone to program something to recognize the waveform automatically?!

    Posted by jeroen | 04/05/2013, 5:09 pm
  8. I read somewhere that it might be possible to extract sounds imbedded in in ancient pottery from when it was created on a pottery wheel.

    Posted by Ron | 04/05/2013, 5:35 pm
    • I think this idea has been debunked. Think for a moment what the frequency response of your fingers’ responses have to sound waves that strike your ears, or even your body as a whole. Very low frequencies indeed MIGHT be recorded in pottery, but nothing audible in the sense of something that would be in the range of human hearing.

      Posted by Brian Kutscher | 04/08/2013, 1:02 pm
  9. where’s the drop??

    Posted by steinbach | 04/05/2013, 5:43 pm
  10. I believe the earliest true “paper recordings” date between 1857 and 1860.

    http://www.nytimes.com/2008/03/27/arts/27soun.html?_r=0

    Posted by John L | 04/05/2013, 5:49 pm
  11. Reblogged this on apreshative and commented:
    Well this stretches the boundary of reason, Let me get back to you on this one.

    Posted by apreshative | 04/05/2013, 9:38 pm
  12. Reblogged this on Adventures in Geekism and commented:
    Wow. Visual to audio, eh?

    Posted by cjpamadaeus | 04/05/2013, 10:37 pm
  13. Reblogged this on Prometheus Press Books and commented:
    Simple amazing. Sound stored in Books, in the late 1800’s. I am blown away.

    Posted by caitlinsumner15 | 04/05/2013, 11:21 pm
  14. If you could calculate the bleed of the ink into the paper and adjust for the tooth of the paper it would clean up a substantial amount of the noise too.

    Posted by ash | 04/06/2013, 1:28 am
  15. Reblogged this on Life, Death and Everything in Between and commented:
    Wow!

    Posted by nalun | 04/06/2013, 4:25 am
  16. That’s seriously incredible! I would’ve never thought there’d be enough information in the book’s image to reconstruct the audio.

    Posted by timothy055 | 04/06/2013, 5:23 am
  17. Damn fascinating. The Big Bang will one day be rebroadcast through emerging technology. Thanks to anyone who has brought these early images to my ears–through reproduced photography from the 1800s. One day we’ll recover the still diminishing sound waves of Lincoln delivering the Gettysburg Address, and hear how high pitched his voice really was.

    Posted by Dennis Flannigan | 04/06/2013, 10:59 am
    • I heard a big bang recreation just the other day on CNN.com. No joke, a scientist took data and converted it to a sound file. It didn’t sound anything like I expected it to sound.

      Posted by Brian Kutscher | 04/08/2013, 1:00 pm
  18. I have also had difficulty finding the software that was used on the web. Does it exist “out there”?

    Posted by Brian Kutscher | 04/06/2013, 2:14 pm
  19. It seems you used an extraordinary lengthy and convoluted process to generate barely audible sound. Wouldn’t writing some software to do this be far easier and produce greater fidelity?

    Posted by Ken | 04/06/2013, 2:40 pm
    • Considering the signal was extracted from a piece of paper with limited spatial resolution, it is remarkable that the signal to noise ratio is as high as it is. The resolution of the scan of the page probably also contributed to some of the “surface” noise in the recording. Audio noise reduction software exists and it is my guess that it WAS used judiciously (not over-doing it) with this recording.

      Posted by Brian Kutscher | 04/08/2013, 12:58 pm
  20. This is brilliant.

    Posted by Kurt Rex Cooper | 04/07/2013, 2:39 am
  21. The concept is at least 11 years old now. http://www.phys.huji.ac.il/~springer/DigitalNeedle/

    Posted by Neal B. Scott | 04/08/2013, 9:46 am
  22. I had predicted this several years ago about how it could be done, (about 2000) however I never had the funds to pursue this, so it did not get any more than a thought. It is nice to see that I was correct!!!

    Posted by Shawn Borri | 04/19/2013, 12:38 pm

Trackbacks/Pingbacks

  1. Pingback: Extracting Audio From Pictures (Summary) « media preservation - 06/20/2012

  2. Pingback: Extreme crate-digging « Lemonade… - 06/21/2012

  3. Pingback: Des images qui parlent « Réjean Beaucage - 06/21/2012

  4. Pingback: Fem fine ting (god helg!) | Brennma.net - 04/05/2013

  5. Pingback: Extracting Audio from Pictures | media preservation | geistwc - 04/05/2013

  6. Pingback: Extracting Audio from Pictures!! | Tamara Temple's Blog - 04/05/2013

  7. Pingback: Audio from Pictures | Paper Caves - 04/05/2013

  8. Pingback: Extracting Audio from Pictures | + ART - 04/05/2013

  9. Pingback: Extracting Audio from Pictures | Bydio - 04/05/2013

  10. Pingback: Listening to Records That No Longer Exist~ from Slate - 04/06/2013

  11. Pingback: Extracting Audio from Pictures | orangelearnof - 04/07/2013

  12. Pingback: Does This Count As Digital?!!! I Think It Must.... - 04/07/2013

  13. Pingback: Audiobook Blog – Audiobooker, by Mary Burkey – Booklist Online » Blog Archive » News you can use - 04/08/2013

  14. Pingback: The Russell Bulletin | Getting audio from pictures of records - 04/08/2013

  15. Pingback: How to Turn a Paper Image of a Record Into a Beautiful Music | Smart News - 04/09/2013

  16. Pingback: Extracting Audio from Pictures | + ART - 04/09/2013

  17. Pingback: Extracting Audio from Pictures | Robot Pirate Ninja - 04/10/2013

  18. Pingback: + ART » Extracting Audio from Pictures - 04/11/2013

  19. Pingback: Сcылки и цитаты #80 : Тысячазнаковбезпробелов - 04/14/2013

  20. Pingback: ATG Hot Topics of the Week | Against-the-Grain.com - 04/16/2013

Leave a reply to Brian Kutscher Cancel reply