"Odyssey"

"Series-of-dumb-decisions-that-surprisingly-result-in-working-software"

VastImg

vastimg.herokuapp.com/?http://some/file.gif

vastimg.herokuapp.com

Let's keep it SFW

lots of GIFs

totally safe for work

Any particular Tumblr

• • Lots of GIFs
• • Identified SFW/NSFW

GifCity

GifCity

Tumblr

benschwarz.github.io/gifcity/?t=tumblrs,go,here

But we can do more

synching

music

DJGif

github.com/geelen/djgif

djgif.com (soon)

Tumblr

Rdio

<img src="probably_cats.gif">

var img = new Image;
img.src = "definitely_cats.gif";

Now what?

✖

Why not video?

var video = document.createElement('video');
video.autoPlay = true;
video.loop = true;
video.playbackRate = 1.5;
video.src = 'converted.gif.mp4';

Convert GIFs to video

✖

an animation

series of frames

Put the frames in the DOM

<div data-frame="0">
  <img src="frame0.gif">
  <img src="frame1.gif">
  <img src="frame2.gif">
</div>

Show one at a time with CSS

[data-frame] img {
  opacity: 0;
  position: absolute;
}

[data-frame="0"] img:nth-child(1),
[data-frame="1"] img:nth-child(2),
[data-frame="2"] img:nth-child(3) {
  opacity: 1;
}

Cycle through with JS

var animate = function() {
  var currentFrame = calculateFrame();
  framesElement.dataset.frame = currentFrame;
  requestAnimationFrame(animate);
}
animate();

Break GIF into frames:

brew install gifsicle
gifsicle -e cats.gif
# outputs cats.gif.001 cats.gif.002 etc

• • Download from Tumblr
• • Split apart on filesystem
• • Serve... somehow?

break apart a GIF

browser?

github.com/kripken/emscripten

• • Stubs out the file system
• • Generates a lot of JS
• • Isn't clever about using Browser APIs

✖

What time is it?

What's in a GIF?

http://www.matthewflickinger.com/lab/ whatsinagif/bits_and_bytes.asp

header

frame frame frame

footer

header frame footer

header frame footer

header frame footer

✔

var blob = new Blob(
  [ buffer1, buffer2, ... ],
  {type: 'mime/type'}
);
img.src = URL.createObjectURL(blob);

var header = gifData.slice(0, startOfFrames);
var footer = gifData.slice(-1);
for (var i = 0; i < indices.length; i++) {
  var frameData = gifData.slice(
    indices[i],
    indices[i+1] || endOfFrames);
  var blob = new Blob(
    [ header, frameData, gifFooter ],
    {type: 'image/gif'});
  frames.push(URL.createObjectURL(blob));
}

Binary in JS!

var xhr = new XMLHttpRequest();
xhr.open('GET', 'cats.gif', true);
xhr.responseType = 'arraybuffer';
xhr.onload = function () {
  this.reponse; // BINARY GIF MADNESS
}
xhr.send();

Same Origin Policy!

CORS on the server

Proxy the content behind a domain you control

Run your own proxy

Use a CDN for evil

CloudFront

djgif.com/*.gif

media.tumblr.com/*.gif

djgif.com/*.*

s3.amazonaws.com/djgif/*.*

We can pull down arbitrary GIF data from Tumblr and split it into frames. Now it's time to look at playback.

Playback

var animate = function() {
  framesElement.dataset.frame =
    calculateFrame( ... );
  requestAnimationFrame(animate);
}
animate();

A component!

<x-gif>

geelen.github.io/x-gif

<img src="images/jump.gif">

<x-gif src="images/jump.gif">

<x-gif src="images/jump.gif">

<x-gif src="images/jump.gif" speed="2">

<img src="images/waves.gif">

<img src="images/vectors.gif">

• ✔ Polymer element
• ✔ Angular directive
• • React component
• • Ember component

geelen.github.io/x-gif

The GIF tag the internet deserves

Beat Detection

Web Audio API

developer.mozilla.org/
en-US/docs/Web_Audio_API

var audioContext = new AudioContext(),
  source = audioContext.createBufferSource(),
  analyser = audioContext.createAnalyser(),
  speakers = audioContext.destination;
  
audioContext.decodeAudioData(mp3data, (buffer) => {
  source.buffer = buffer;
  source.connect(analyser);
  analyser.connect(speakers);
})

// Once the music is playing
var frequencyData = new Uint8Array(1024);
analyser.getByteFrequencyData(frequencyData);
frequencyData[19]; // Intensity of ~440Hz

navigator.getUserMedia({audio: true}, (stream) => {
  var source = audioContext
                  .createMediaStreamSource(stream);
  source.connect(analyser);
  analyser.connect(speakers);
})

24ways.org/2013/
make-your-browser-dance

var frequencyData = new Uint8Array(1024),
  beatDetector = // TODO: this.
  
// called within a requestAnimationFrame loop
var calculateFrame = function() {
  analyser.getByteFrequencyData(frequencyData);
  beatDetector.update(frequencyData);
  return beatDetector.howFarThroughBeat();
}

✖

developer.echonest.com

http://developer.echonest.com/
  api/v4/song/search?
    api_key=API_KEY&
    format=json&
    results=1&
    artist=weezer&
    title=el%20scorcho&
    bucket=audio_summary

• • Search ~ 1kb
• • Analysis ~ 300kb
• Both require private API key

NodeJS Micro Service!

app.get('/search', (req, res) => {
  var url = searchUrl(req.query.artist, req.query.title)
  request.get(url, (err, data) => {
    var track = JSON.parse(data.body);
    var summary = track.response.songs[0].audio_summary;
    request.get(summary.analysis_url, (err, data) => {
      var analysis = JSON.parse(data.body);
      delete analysis.segments;
      delete analysis.tatums;
      track.response.songs[0].analysis = analysis;
      res.send(200, track);
    });
  });
});

http://djgif-echonest-proxy.herokuapp.com/
  search?artist=weezer&title=el+scorcho

• ✔ 55kb
• ✔ No API keys on client

echonest-proxy

<x-gif src="images/explosion.gif" sync>

var xGif = document.querySelector('x-gif'),
  beats = metadata.beats, //from Echonest proxy
  animationLoop = () => {
    // calculate these three:
    xGif.clock(beatNr, beatDuration, beatFraction);
  };
animationLoop();

DEMO

DJGif

Shipped, right?

✖

Side projects

products

• ✔ Binary JS hax
• ✔ Cloudfront proxying
• ✔ Transcoding GIFs to Video
• ✔ Web Audio Analysis API
• ✔ Beat synchronisation
• ✔ Polymer & Web Components

• • Beat detection algorithms
• • RXJS + React
• • GIF mood detection

Thanks!

geelen.github.io/a-gif-odyssey

DEMO