PDA

View Full Version : "Salvation" - Visual OpenGL design



Trey
09-05-2003, 11:35 AM
Just wanted to let people know about a project I've been working on for a few years now. The central idea is to redefine graphics programming and turn it into a visual, interactive, realtime activity.

Graphics animations are created not by endless lines of code, but by visual "scene graphs" that represent the drawing processes.

Here is a screenshot showing the recently incorporated ARB_fragment_program compiler:
http://www.treyharrison.com/tshader.jpg

Being realtime and interactive, "Salvation" has primarily caught the interest of video artists and "vjs", but I think game developers and other developers in general may have various uses for it.. to quickly construct previews / demos of their graphics content, to assist with development of vertex and pixel shaders, and perhaps more in the future.

Those with an interest in the technology (beta testers, critics, people with suggestions, etc) are encouraged to contact me at trey@u.washington.edu or to reply here on the Opengl forum.

thanks for listening.

trey
ps. some preliminary designs created with "Salvation":
http://www.treyharrison.com/loops http://www.zygotia.com/videos.html

^Fishman
09-06-2003, 12:06 AM
Impressive.

[This message has been edited by ^Fishman (edited 09-06-2003).]

knackered
09-06-2003, 04:14 AM
I can't think of one content artist I know who would be able to understand 1% of those images...
Nice idea in theory, but in practice you're just replacing coding-by-keyboard with coding-by-mouse....either way, you have to be a coder to do it.

Trey
09-06-2003, 07:48 AM
Originally posted by knackered:
I can't think of one content artist I know who would be able to understand 1% of those images...
Nice idea in theory, but in practice you're just replacing coding-by-keyboard with coding-by-mouse....either way, you have to be a coder to do it.

I agree with what you say about having to be a coder to do it. It is programming by mouse, yes - so of course anyone who uses it is going to be coding, visually, at a high level. I don't think knowledge of C is required, though, to drag the mouse from one little triangle to another. When you say "have to be a coder to do it", are you implying that you would have to know one of the written languages before you could understand the visual representation of a program? Do you think that the fundamental concepts of programming are locked up inside the syntax of a written language?

I believe Salvation can introduce fundamental programming concepts to people who have absolutely no programming experience or knowledge - and can do so in a realtime, interactive way. It can also do it very quickly, by avoiding all of the educational overhead involved in teaching the syntax of a more formal written language. I don't have wild dreams of completely replacing traditional coding with visual coding - thats a ridiculous proposition. But I do think that visual coding has its place.

You make a valid criticism regarding the understand-ability of that screenshot I posted. A lot of those modules are 1:1 mappings of OpenGL functions, so understanding that screenshot requires knowledge of the OpenGL state machine. This is not how I envision the final product, and certainly not how it will be presented to first time users.

In the same way coders use abstraction to group their code into functions, classes, multiple files, etc, so will the OpenGL modules be grouped and effectively hidden a layer or two below the level at which the average visual artist works. Their first tutorial might involve connecting a "RotatingCube" box to the "Screen" box, and tweaking the "Angle", "Color", or "Size" parameters.

I don't know if you intend for your comment about artists not understanding that screenshot to generalize to artists not understanding coding-by-mouse, but there are already many who do so, with other programs that have been around for a long time. The Max/MSP tool was originally a visual programming tool for sound artists / musicians, and it recently was upgraded with OpenGL capabilities via an "external" called Jitter. I believe they are working on a Windows port right now but I'm not sure whether its been completed or not.

thanks for the response,

trey

knackered
09-06-2003, 10:07 AM
Perhaps you have the priviledge of working with brigher artists than I do! http://www.opengl.org/discussion_boards/ubb/wink.gif
BTW, it looks like a fantastic learning tool for programmers, but do non-programmers have the time or inclination for this? What's your market research shown?

P.S. I don't mean that your product is rubbish, just still don't get who it's aimed at.

Trey
09-06-2003, 01:12 PM
Originally posted by knackered:
Perhaps you have the priviledge of working with brigher artists than I do! http://www.opengl.org/discussion_boards/ubb/wink.gif
BTW, it looks like a fantastic learning tool for programmers, but do non-programmers have the time or inclination for this? What's your market research shown?

P.S. I don't mean that your product is rubbish, just still don't get who it's aimed at.

If your point is to say that most people don't have a need to program their computers, thats fine - its true and valid.

Salvation is aimed at people like myself: "sound/video artist/programmers". A niche group, certainly, but hey, its the one I'm part of.

I thought the best way to find other "sound/video artist/programmers" would be to ask the sound/video artists "hey what do you think of programming?" and to ask the programmers "hey what do you think of sound/video art?"

Which one are you, artist or programmer, and what do you think of the other? http://www.opengl.org/discussion_boards/ubb/smile.gif

trey

[This message has been edited by Trey (edited 09-06-2003).]

Korval
09-06-2003, 01:50 PM
Programmers are artists. Only our medium of expression is different.

Trey
09-06-2003, 07:06 PM
Originally posted by Korval:
Programmers are artists. Only our medium of expression is different.

Thanks for the comment. Do you have anything to say about the software I've been writing?

Jan
09-07-2003, 04:16 AM
Well, i think itīs a great tool for stuff like MTV does it. I mean all those short clips which are shown in between, or to anounce advertisement, etc.

Certainly an artist does have to know the basics of programming, but hey, i know a lot of people who are very creative and have basic programming skills. Maybe "todays" artists are not used to programming, but in 5 or 10 years everybody will have learned the basics of programming at school.

And to learn stuff like Photoshop or 3D Max etc. you need years. Those are an artistsī tools. Why should one expect, that learning this tool takes less time.

Everyone does what he is best at. If an artist is not able to learn such stuff, than he wonīt do it. Others will.

Just my 0.50 Euro.
Jan.

Mihail121
09-07-2003, 01:26 PM
Well damn the man said it took years,the screenshots look cool(amazing is more right),he gives reasonable arguments and we behave like a bunch of loosers.It's very good looking pal and i think in some place it can do a very cool job.I consider the screenshots superb.The interface and etc. is just uniqe.Nice job!

SirKnight
09-07-2003, 04:14 PM
Neat. So....when can we try it out? http://www.opengl.org/discussion_boards/ubb/biggrin.gif


-SirKnight

Trey
09-07-2003, 11:34 PM
Originally posted by SirKnight:
Neat. So....when can we try it out? http://www.opengl.org/discussion_boards/ubb/biggrin.gif


Right now! Anyone who wants to play should email me and I will give out beta-testing signup info. Minimum requirements: WinME/2k/XP with 1.3ghz CPU w/NVidia Geforce 2 or better. Preferred requirements: 1.3ghz CPU w/NVidia Geforce FX (or one of the ATI radeons that support arb_fragment_program)

thanks for your interest =)

Trey

tfpsly
09-07-2003, 11:56 PM
Originally posted by Korval:
Programmers are artists. Only our medium of expression is different.

=)
Makes me remember why I likes so much the demoscene and 3d programming at first : I just sux at pencil drawing http://www.opengl.org/discussion_boards/ubb/biggrin.gif

Trey
09-08-2003, 08:33 AM
Originally posted by tfpsly:
=)
Makes me remember why I likes so much the demoscene and 3d programming at first : I just sux at pencil drawing http://www.opengl.org/discussion_boards/ubb/biggrin.gif

Hey, I'm VERY GLAD you mentioned the demoscene because that scene (which I've never been personally involved in, sadly) is perhaps the greatest inspiration behind the designs I create with "Salvation".

It is not the perfect platform on which to create "demos" at this time, but since the beginning I've always been asking myself "What can I add to this software that would make it easier to use for creating "demos", or something like them?"

trey

V-man
09-08-2003, 08:53 AM
Trey,

I would like to know how the tool is suppose to function (what does the user have to do, and what happens when he is finished). I would be happy to test it for you, as long as it can do something meaningful.

I have used Flash, and I think more or less it`s not bad, but the tools it offers are very "basic". It has a lot of perculiarities as well.
It is somewhat aimed at the part-artist and part-programmer type.
I think you can benifit by trying Flash, if you haven`t.

Trey
09-08-2003, 11:02 AM
Originally posted by V-man:
Trey,
I would like to know how the tool is suppose to function (what does the user have to do, and what happens when he is finished). I would be happy to test it for you, as long as it can do something meaningful.


Salvation is divided right now into two sections: the "Screens" and the "Workspace". The workspace is where the user constructs ("writes") his or her program. The screens are where the program output is viewed.

Construction of a program is achieved by selecting one of the many "modules" I've already programmed, setting it down somewhere in the workspace, and connecting it to other modules. One of the essential modules you must use to get any results is called "GLDisplay" and it is used to connect the output of your programs to the Screens. (you can see it in the screenshot mentioned in the first post). The "module" concept is such that C++ programmers can create their own modules and incorporate them into Salvation without an excessive amount of pain (close to 100% of the module code thus far resides in .dll files). Module authors do not even have to worry with writing any GUI code, in most cases.

Right now there is only one available output screen, "Screen0", and it can be either a window, fullscreen, or fullscreen on another monitor (if your card supports it). You can capture the output of Screen0 to a .avi file at any time, and there is also a way to capture seamlessly looping .avi files, as seen here:
http://www.treyharrison.com/loops

So, "Salvation" is (right now) used to create graphical animations. You can store them to an .avi or view them (and edit/design them) in realtime. Also you can configure Salvation to be your Windows screensaver, which is nice =). I've also written a Fruityloops Instrument plugin version of Salvation which allows the visuals to synchronize with and be controlled by various parameters from the Fruityloops program. I haven't used the Fruityloops plugin much lately, but I did use it to create these:
http://www.zygotia.com/videos.html

Here's a simple design you might try if you were first getting started:

Create a GLClear module and a GLDisplay module. Connect the output of the GLClear module to the input of the GLDisplay module. This produces a nice black Screen0 (but Screen0 was already black so you haven't achieved much). To get a colored Screen0, you would create a Color module and connect its output to the color input of GLClear. The knobs on the color module would then define what color GLClear uses to clear the screen.

Am I making any sense? http://www.opengl.org/discussion_boards/ubb/smile.gif

trey

JustHanging
09-10-2003, 03:14 AM
Not all programmers are artists IMO, just like not everybody holding a brush is an artist.

Anyway, your project looks ok, can you do anything non-psychedelic with it? About the user interface, there's way too much information on screen, you might want to hide it somehow. Easiest ways would be collapsable/expandable nodes or seperating the attributes from the graph, showing the attributes of the selected node in a seperate window.

It could be a good idea to have a look at Maya (PLE for free) for some ideas, the hypergraph and shader networks bear a lot of similarity to your project, altough the level of abstraction is a bit higher.

-Ilkka

Trey
09-11-2003, 09:19 AM
Originally posted by JustHanging:
Not all programmers are artists IMO, just like not everybody holding a brush is an artist.

Anyway, your project looks ok, can you do anything non-psychedelic with it? About the user interface, there's way too much information on screen, you might want to hide it somehow. Easiest ways would be collapsable/expandable nodes or seperating the attributes from the graph, showing the attributes of the selected node in a seperate window.

It could be a good idea to have a look at Maya (PLE for free) for some ideas, the hypergraph and shader networks bear a lot of similarity to your project, altough the level of abstraction is a bit higher.

-Ilkka

The character of the animation generated by the software is entirely up to the person using it. Salvation is not strictly aimed at the production of "psychadelic" visuals.

To be fair, I admit that the library of modules I've made thus far does limit the designs to a "abstract" or "psychadelic" space. It takes time to code all this you know =). Some future ideas: character design/import (from other 3d softwares like the ones you mentioned), generic scene design/lighting, scene transitions, etc.

Regarding information/parameter/attribute visibility:

This is up to the user and how they choose to organize their work. Modules can be grouped and abstracted with containers, and module GUI's can be customized to show all, some, or none of the module's available parameters.

A seperate "Attributes" window is a fine idea, although I personally find it easier to understand a design by looking at a single integrated view of modules, parameters, and signal routings. My approach is to show them all the parameters first, then let them hide what they want. Yours is to show them no parameters first, and have them click to see what they want. Both approaches are possible given the framework, I just don't see myself adding a properties window any time soon http://www.opengl.org/discussion_boards/ubb/smile.gif.

thanks for the comments,

trey

Trey
09-11-2003, 10:29 AM
Originally posted by JustHanging:
Not all programmers are artists IMO, just like not everybody holding a brush is an artist.

Just wanted to share some thoughts about this one...

If your point is that not all programmers can draw, its a valid one.

But I don't think the "artist vs. programmer" question ends there. Someone else in this thread hinted at a higher level similarity, which I will try to describe:

Both "Artists" and "Programmers" spend their time consciously focused on two things 1) the discovery of ideas 2) the manifestation of those ideas through their medium of choice.

"Artists" traditionally work with physical materials.. brushes, oil paints, water colors, pencils, etc. Some of their ideas: a new image to draw, a new technique to use with brushes, a new shading technique, a new pottery technique, etc.

"Programmers" work, obviously, with computers and digits (numbers). Some of their ideas: a new method for sorting numbers, a new method for visualizing numbers, a new method for representing photographs or sound in terms of numbers, etc.

"Salvation"(tm) is designed to be a convenient, flexible, and expandable platform for the manifestation of new programming ideas, the storage and retreival of old programming ideas, and the combination/interaction of both new and old programming ideas.

With "Salvation", new programming ideas are given life in one of two ways: 1) Using the high level visual language and existing library of modules to create a design 2) Using a lower level language (C++, C, etc) to create a new module.

Right now all of the modules work closely with OpenGL (and the data types associated with graphics) - but nothing about Salvation prevents me from creating and adding modules to do other things (send and receive information over a network, send and receive audio data to and from a sound card, send and receive controller information from a midi device, etc).

Have I blabbered enough? http://www.opengl.org/discussion_boards/ubb/smile.gif

trey

[This message has been edited by Trey (edited 09-11-2003).]

[This message has been edited by Trey (edited 09-11-2003).]

dbugger
09-11-2003, 11:06 AM
Hi! Great idea IMO! What you are doing is a kind of scene-graph modeller, right? I believe such programs will be far more common in the near future as programming is getting more and more complex. Soon all we have to write are some adjustments to the underlying code, like we do with assembler today. The graphical interface is just another abstraction layer, necessary to write even larger programs.

That said, I don't think tools like that will be for artists, but for programmers.

Trey
09-11-2003, 03:55 PM
Here is a screenshot for the people who were complaining about how theres too much information on-screen at once:
http://www.treyharrison.com/minimal.jpg

With the software as it is now, this is the simplest possible viewing of any design. In fact, all designs simplify down to the representation seen there.

Due to the lack of visual stimulation in that screenshot, I must also provide this, a 256x256 avi (Divx5) of "Bluepool":
http://www.treyharrison.com/minimal.avi

cheers,

trey

JustHanging
09-12-2003, 11:16 AM
By my point about not all programmers being artists I just meant that if you're writing a program to realize a given protocol, for example, I wouldn't call you an artist based on that. An analog would be someone painting a house.

Both are honorable professions, of course.

-Ilkka

Jan
09-12-2003, 02:17 PM
It still depends on how you paint a house http://www.opengl.org/discussion_boards/ubb/biggrin.gif http://www.opengl.org/discussion_boards/ubb/biggrin.gif http://www.opengl.org/discussion_boards/ubb/biggrin.gif