See Hear Project

Our first project of 2nd year is to make an animation based on a 15 second clip of soundtrack. To start off, we need to put our given sound-clip into some abstract visuals that showcase the music's elements. Abstract is pretty out of my comfort zone, so I need to challenge myself with this and experiment! Another challenge is that the soundtrack is taken from cartoons, which in turn makes me think of cartoons when I hear it. So I need to distance from that initial connection and pretend like the soundtrack piece is completely alien to my ears! First, some research done over the week:

Research and Inspirations:

Researching for this project has been a good excuse to look into the combination of sound/music and visuals in film generally. When starting to think about examples to help with this project, the most obvious reminder was Disney's Fantasia - where animation is set to classical music. The film-makers and animators were working in the same way as us; creating an animation set to an already composed piece of music, so they had to make the animation fit with and represent the music. I first thought of the Fantasia 2000 animation to Gershwin's "Rhapsody in Blue" - one of my favourite ones as the action, the colours and style match the music perfectly in a 1930s New York setting. I particularly love the start when the city is being drawn out by a simple line (on a blue background of course) that follows the clarinet solo, and when the clarinet plays an ascending scale it is combined with that line suddenly shooting into the air to form the first skyscraper (unfortunately I couldn't find the whole animation online!):



Although what Disney created was complex with narratives and characters etc - and what we are trying to do at the moment is create something more abstract to represent the sound - this was useful to look at again. As I said already, the film-makers were trying to do the same thing as us - harmoniously match animation to music! There is also a more abstract animation from the Fantasia 2000 set to Beethoven which plays around a lot with colour and shapes:

 

Another great example was the part in Pixar's "Ratatouille when Remy the rat is tasting food, and his 'feelings' I suppose, are being represented with animated shapes and colours or "fireworks". The animation matches the jazz music and both are meant to represent the food he is eating:




I was also reminded of film credits, particularly those from animated films of course. End or title credits often have good 'music and animation' sequences. I particularly like the jazzy title credit from Moster's Inc where the action is definitely tight with the animation (with things like doors slamming and the words forming etc). The music is very much in time with what the instruments are playing:



Ok, now trying to think away from Disney/Pixar but continuing on film credits and titles...
I particularly like the title sequence for "A Fistful of Dollars" with music by Ennio Morricone. Morricone is a good composer to listen too as well, not only for his awesome scores, but for his use of sound-effects as part of the music - so in the title for "A Fistful of Dollars" sounds like whistling and crack/snaps are part of the music, along with the gun-shots and horses galloping. The horses' galloping is pretty much in time with the music also. The visuals brilliantly match the music in time and with actions to coincide with the gunshots. The visuals match the mood and atmosphere created by the music too with the dark, bold colours:



The James Bond films have pretty good title sequences also, first off being "Dr. No". Again it uses some interesting sound affects to start off with the classic Bond gun sequence - sound affects that are completely unrelated to what is being shown (for example, what sounds like a xylophone/glockenspiel slide to represent the gun-barrel appearing into view). Then into a somewhat abstract colour and circle-shape display in time to the music for the first part of the title:



Probably my favourite of the Bond sequences however is that from "Casino Royale". I think it's a great animated sequence, which again matches the song well, and is full of animated patterns and symbols:



Something a bit different now; I was also reminded of the climatic scene from "Close Encounters of the Third Kind" where the aliens are trying to communicate with Earth via light and music. It's not as relevant to what we are doing, but I really like this scene with the combination of music notes and the lights of the space-ship -  I think this combination is very important to the scene, it wouldn't be as effective with either just the music or just the lights:




Sketches and Initial Ideas:

Once I had my clip of soundtrack, I first made notes of the characteristics of the sound. I found that the soundtrack was in three parts; the first part is a descending pizzicato scale which sounds hurried and bouncy, but cautious maybe. I imagined calm but with unsettlement. It reminded me of going down a staircase due to the descending scale, but this was to literal so I listened more and tried to think of other visuals that came to mind. It's bouncy pizzicato made it bubbly. Not exactly smooth, but wavy or bubbly. I immediately imagined blue tones, getting deeper and darker as the notes got lower. Here are some of my sketches done over the week:

These are my first sketches of the sound. On the right-hand page is a sketched sequence for how the action/movement might play out.

On the left-hand page you can see I was still thinking of this idea of stairs, but then changed to a more abstract, bubbly pattern (next page), with the green flare as the start of the music's middle-part...

With an ascending flute trill-thing, the second or middle part is a light, airy, more optimistic and more curious sound than the first part. Very quick, bright and sharp. You could say 'short but sweet', as it quickly turns to a two-note glockenspiel or xylophone, then a low flute note that is held and feels like it settles calmly. I immedietely imagined something like a feather flying up then floating calmly for a few seconds, with more vibrant colours like greens and yellows, but still harmonious with the blue:

The soundtrack ends with a gun-shot sound effect that cuts right through the calm, settled flute note. It is very brash, big and aggressive. As the gun-shot sound prevails it seems to get sharper and thinner, and sounds further away. I imagined the gun shot coming literally 'out-of-the-blue' and being completely contrasting to the previous atmosphere:

After my attempts to characterise each part of the music, and sketch out the possible action, I went to thinking how all 3 parts could connect to form the 'timeline strip' (to be created later on...)

We also did some work with dope sheets, which was very helpful for creating a plan for the timeline. I think dope sheets are definitely something I would like to use in future, especially for any work that is sound heavy, as a guideline and reference. It was good to combine both my own 'spontaneous' artwork drawn just from listening to the music and the dope sheets which pin point where all the sounds actually are in the track...