A script that converts a movieclip into a raster image?

Moderators: Disciple, zlovatt

Post Reply
Spacehopper2000
Posts: 1
Joined: May 16th, 2007, 10:29 am

Hello,

I've been looking all over for a script (for AE or Maya) that converts a movie into a customated raster.

More exact; takes the avarage luminosity for a square in a grid (in each frame of a movie) and convertes it into a predefined raster image or animation.

See http://www.youtube.com/watch?v=N5XVeENmLMk and you'll get what I mean. Anyone that can help me out with this one?

Thanks!
Dan Ebberts
Posts: 320
Joined: June 26th, 2004, 10:01 am
Location: Folsom, CA
Contact:

What a fun project. I think it would have to be an AE CS3 script that would construct the appropriate sampleImage() expression and harvest the results to decide which image to place at each location in the grid for each frame. It would probably take a long time to run, but it would be very cool.

Dan
nerken
Posts: 14
Joined: October 16th, 2006, 3:40 am
Location: Barcelona Spain
Contact:

sure, it-s sampleImage! you can pick it from any pixel

very cool
nerken
Posts: 14
Joined: October 16th, 2006, 3:40 am
Location: Barcelona Spain
Contact:

OH, sorry, but when i wanted to do something similar i used the mosaic effect and mask it with a layer grid.
RoaneSky
Posts: 3
Joined: May 26th, 2007, 8:40 pm

Yeah, I'm trying to do this too. I did a version in Flash using Actionscript's bitmapdata. I've been coding with that for a few years. But I want to output to high quality video, so i tackled AE. I've been using AE for stuff for years too, but not the scripting. Man, I gotta tell ya, the syntax difference is killing me. Even though it's JS, I can't get most things to work right. (could be that I'm expecting too much functionality from it).

Here's a rough outline of what I'm trying to hit on:

Use script to setup an expression:
...position.canSetExpression;
exp = "thisComp.layer(2).sampleImage(transform.position, [2, 1.5]), false, time);";
....position.expression = exp;

This writes the expression, but then disables it because something's wrong. I can see that even though it sets the expression, I don't see a way to get the results back to my script. What it should do is write this expression to each little square layer for the mosaic, then return the sampled value to the script -- then I can set that little square accordingly. These little squares are written out through a loop.

Yeah, I know this would be slow, but rendering wise I'm not in a hurry. I just want this kind of control like I have in my Flash version.

If someone could shed some light on this part, I can probably do most of the other. I'm currenty coding a dummy version using a randomly set [r, g, b, a] array to pretend I'm getting data back. Then, when I figure out this expression part, I can just plug that in and go.

I need a good book on AE CS3 scripting. I have the PDF guide, but man, I hope this isn't ALL of it? Real examples of working syntax helps me more than a guide like this PDF.

Thanks guys.
Dan Ebberts
Posts: 320
Joined: June 26th, 2004, 10:01 am
Location: Folsom, CA
Contact:

I think you just have an extra right paren in there:

exp = "thisComp.layer(2).sampleImage(transform.position, [2, 1.5], false, time);";


Dan
RoaneSky
Posts: 3
Joined: May 26th, 2007, 8:40 pm

hahaha, yep. I see it now. Thanks man. :)

I guess it does return when you get the syntax right. I'll give it a try.

UPDATE:

I tried it and I get "expression result must be of dimension 2, not 4, Expression disabled.

exp = "thisComp.layer(2).sampleImage(transform.position, [2, 1.5], false, time);";
workingLayer.position.expression = exp;

Here's the thing I noticed, transform.position should return x,y,z right? and sampleImage just wants x,y right? I can'f find any really clear explainations anywhere for how this thing works. A head scratcher.
RoaneSky
Posts: 3
Joined: May 26th, 2007, 8:40 pm

For the original poster, here's some info I've discovered.

It will take between 2000 and 5000 layers to create the effect depending on the working resolution and how small you want the mosaic pieces to be.

This takes a LONG time to pack in. I know I said I wasn't in a hurry, HOWEVER, it just took over 15 mintes to do one frame. So I guess that means after a several DAYS! of rendering, you would still only have a second of footage yeah? What a bummer.

And these numbers don't even include getting the sampleimage data or cleaning up the frame to build the next one.

When I did this in Flash, the screen filled with a mosaic instantly. Maybe I need to optimize something. Maybe I need to loop x and y in a nest.

Any thoughts?
nab
Posts: 203
Joined: November 29th, 2005, 3:00 am
Location: Royan
Contact:

That's fun! I concur. :)

I can't imagine a raisonable time consuming script though. It should be ok for small image with very few cells in the grid and couple of different tiles only...on a 5-frames long sequence :lol: (I didn't try actually so I'm bit Off Topic !)

Last month I wrote a little Windows application to produce this sort of 'photomosaic' from a (still) source image and a folder of tiles (other images). I used CImg (c++ opensource library) to deal with image data...it worked well but the overall process was too slow to allow thinking of a 'multiple sources' extension (i.e. a sequence of frames).
When I read your post (Spacehopper2000), I decided to give it a second attempt. This time I used Intel's OpenCV (other c++ library).
The program is now incredibly fast :P. I've added video file (avi) support in addition to the single still image to become a 'Photo/Video Mosaic' tool.
It runs nearly in real time (say 5->20 fps with 1G of ram) for mid sized input/output (dv format) and 10000-cells grid per frame.
I rendered some enormous video file for crash testing purpose, up to 3000 pixels, each frame containing more than a million of cells; it took a minute or so to render the 250 video frames. (I may provide some samples but I don't plan to share it immediately).

Well, to make this long story shorter, although AE scripting is really great it's (very) limited when it comes to intensive image processing (by intensive I mean when you have a very large quantity of data to deal with); in such a case you have to work with other language and tool (c++ plugin to stay in AE, otherwise language of your choice in an external app).
Colin Braley
Posts: 13
Joined: July 14th, 2005, 7:33 pm
Location: VA
Contact:

I have a project in which I did just this...I finished it a few days ago. I used 2 expressions. One expression places the layers in a grid, and the other one uses sampleImage to sample pixels from a movie layer and use this to drive the color parameter of the "fill" effect. The project uses some expression controls and is fairly tweakable. I'll probably put it up on my site sometime in the coming weeks, but if anyone wants it now, let me know.
~Colin
PS - It actually only took about 3-4 seconds per frame for me, and I am using 972 tiles, so it isn't too bad.
Post Reply