Anyone who’s read my latest blog posts will know that I’m a big fan of Mr doobs Three.js JavaScript 3D engine, so I won’t bang on about it any more than I have to. For my latest experiment I decided to mix the HTML5 File API, Three.js and a few images (demo here).
Mr Doob and a little kitten sitting in a 3D tree.
The File API allows a user to upload a file (an image in this experiment) to a web application simply by dragging it onto a designated “droparea”. Once dropped, you can manipulate the data any way you like. The API is able to handle multiple files natively, so just select all the files and drop them onto the area. No need for Flash uploaders or multiple browse boxes. Of course not all browsers support it (IE) and there are a few inconsistencies in the specification; but they will work themselves out over time.
So what does the experiment actually do? Well it allows a user to drag an image onto their browser window which is then “uploaded”; the data is then passed to a canvas element and then this data is passed to Three.js to be plotted in 3D space. The X and Z-axis is plotted in the usual manner to represent the default 2D image using particles; the fun bit comes when you add the Y-axis. This is calculated by taking an average of the RGB values (greyscale) and plotting it accordingly. When using an image of a human face it’s amazing how accurate the 3D plot looks.
Due to the amount of data involved it is recommended that you use small thumbnail images when you try the demo, as larger images equal a larger number of particles. This is an initial version so I’m going to work on a way of dealing with larger images. This will probably mean ignoring a certain number of pixels when looping through the image data. Looking forward, since it can be done with a static image I see no reason why it can’t be done with a video, but that’s a project for another day!
Update: I’ve updated the demo slightly; as Mr.doob pointed out people may have tried to drag directly onto the render area which didn’t work. So now it does :)
Yesterday evening I had the pleasure of attending the Open Web Gaming event #gameon10 hosted by Mozilla Labs and Six to Start. The evening involved five short main talks and numerous lightning talks by various attendees.
Christian Heilmann getting ready for his talk at #gameon10.
There were an interesting mix of speakers, from the big name companies like Google and Mozilla, down to start-ups and independents:
- Christian Heilmann: Very interesting talk on why developers should be developing using new open web technologies. He had some very interesting ideas for Angry Birds too (see below).
- Ernesto Jimenez: A hands on talk on the “Do’s and Don’ts when developing with the HTML5 canvas element. Mainly about things he’d learnt the hard way.
- Rob Hawkes: Rawkets is a simple shooter game Rob developed for a University project that’s really taken off. It uses various open technologies including node.js and Web Sockets on the server side.
- Rik Lomas: Is the line between TV and computer entertainment bluring? With the help of Picklive it certainly is. A fantasy football game played in real time, built on Javascript, jQuery, Strophe and XMPP.
- Paul Truong: Paul’s talk didn’t quite go to plan with a couple of teething problems with the demonstrations, but it was still very informative. I had no idea MacBook Pro’s had a built in accelerometer, and using it for gaming is a stroke of genius (although I can’t see many people tipping their MacBook Pro, it’s a tad heavy for a controller). With the iPad you’re on to a winner.
Paul Truong talking about Processing.js.
I particularly liked Christian Heilmann’s talk as he had some great ideas for Angry Birds if it was built on open web technology. You could very easily allow people to build their own levels, add multi-player support, add a frustration rating for levels and user comments. Adding the social level building aspect to the game takes a bulk of the development time off the game developers; this is exactly what Media Molecule have done with Little Big Planet and they now have 1 million+ levels! As an extra you could setup level building competitions and bring out a version of the Angry Birds based purely on the best custom built levels. Everyone’s a winner!
The evening was great fun and all the speakers were brilliant. My only criticism would be that the talks seemed very rushed. Making the talks fifteen minutes instead of ten would have allowed for a slightly slower pace with a few more demos. Big thank you to Mozilla Labs, Six to Start and all the speakers.
Earlier this week I came across a very interesting site by Mike Kamermans which had a demonstration of the Euler spiral (also known as the Cornu or Fresnel spiral) using processing.js. Very interesting! After a quick read of the wikipedia article (and discovering my Maths is rusty) I decided to create a version of my own using my current favourite toy: Three.js by Mr doob.
There are a few settings available to change so have a play around.
I’ve put together a few user controls using jQuery UI that you can use to change certain aspects of the animation:
- Scaling: Scale along the X and Z-axis.
- Y axis increment: Add some depth along the Y-axis.
- Selective particle freeze: Exclude certain particles from the animation.
- Y-Axis target: Adjust the camera target in the Y direction.
- Freeze all particles: Exclude all particles from the animation.
It’s amazing what a simple set of mathematical equations can achieve. In terms of web technology, my next step is to add a version using the HTML5 Web Worker API (see update below) that will take the calculations off the UI thread allowing for a more responsive browser UI when animating a large number of particles.
Note: Try the demo using both Firefox and Chrome; you can really see the difference between the Tracemonkey (FF) and V8 (Chrome) JavaScript engines.
Why not have a play with the demo and let me know what you think!.
Update: I’ve now added a version using Web Workers located here. A couple of things to note from this demo:
- Web Worker threads run asynchronously below the main JavaScript thread, so be careful when trying to use the returned data. Be sure that the data exists by using the ‘worker.onmessage’ callback before you fire any dependencies. If you don’t then expect quite a few “Undefined” error messages!
- While using the Workers for the animation data I’ve had to limit the animation to 10fps (down from 60fps). Surprisingly enough browsers don’t like it when you spawn 60 worker threads per second :)
I’ll be sticking to using the main thread for calculation in the future as there’s no advantage to using Web Workers, but it was interesting proof of concept.
Last night I decided to revisit an experiment I wrote earlier in the year: Using Image Data Inside the HTML5 Canvas Element. It involved loading 2 – 3 images and swapping out the image data depending on the mouse position. I ran into a slight issue though, it wasn’t working:
1
| uncaught exception: [Exception... "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIDOMCanvasRenderingContext2D.drawImage]" nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)" location: "JS frame :: http://nooshu.com/explore/canvas-image-data/js/ci.js :: anonymous :: line 27" data: no] |
Not good! I double checked the code on my local machine and it worked. Very odd! I noticed that it wasn’t working when you first hit the page but was on page reload. Then it struck me; it must be a caching issue. I was correct.
It turns out that the JavaScript was firing before all the images had fully loaded.
1
2
| context.drawImage(lakeImage, 0, 0);
var originalLakeImageData = context.getImageData(0,0, iWidth, iHeight); |
The above lines were trying to access ImageData that didn’t exist yet and the browser was falling over because of it. Doh! Luckily I’d used jQuery for a couple of functions in the experiment so I quickly wrapped the code in a load method, problem solved!
1
2
3
| $(window).load(function(){
//Your code here
}); |
This method waits until the whole page has loaded (including images) before the code inside it is fired, guaranteeing the image data is there to be manipulated when needed. View the updated demo.
Mr Doob has created a superb JavaScript 3D Engine in three.js. Screenshots from my 'Bowl' demo.
I’ve been watching the development of three.js by Mr Doob for a couple of months and have been desperate to experiment with it. With Firefox 4 well into the beta stage I decided to bite the bullet and do a little Audio Data visualisation in 3D!
I’ve dabbled with audio visualisation and the HTML5 Audio Data API in the past, so I’m pleased I’ve had a chance to use it again and see what’s new. Note: My previous audio visualisations using canvas no longer work due to specification changes in the API. A pitfall of using an API that’s still in the draft stages I guess.
I’ve put together 5 simple demo’s all based around the same theme showing some of the options available in three.js. My favourite is “bowl”, pretty colours galore!
- Circle: Simple circle, zoom in / out using the mouse wheel.
- Cubes: Same as circle only using the cube primitive.
- Cylinder: Using the cylinder primitive with a floating camera.
- Sinewave: Rotated sinewave (looks a little like a chandelier).
- Bowl: A combination of the above demos to create an interesting 3D bowl effect.
Please note these demos will only work in Firefox 4. The Audio Data API is still a draft specification and Mozilla is the only browser vendor to have added it to date. I’m looking forward to it being added to Chrome (Webkit), with its superb V8 JavaScript engine. The demo’s really are just the tip of the iceberg for both three.js and the Audio Data API. It’s certainly an area of the HTML5 specification to keep your eye on.
The music track used in the visualisation is called “Sad Robot” by a group called “Pornophonique”. Very catchy tune that you can download from Jamendo.
Update: It looks like enabling the Google Page Speed Apache module seems to have broken the demos. I’ve disabled it now so it should hopefully fix it (fingers crossed!).
