simple CGA shader

Hi I’m new here and I’m trying to get into shaders
It’s a very fascinating thing, in particular I need it for game development

I use it in game maker studio professional which allows the using of glsl es language
and I started reading the hard manual and following a lot of video tutorials online

actually I’m able to change colors, channel colors of the original color, make an average to draw in grayscale and using the uniform variable to make a bridge from the shader and the object that I need to be drawn with the shader (or everything if I draw the shader to the surface)

actually I’m developing a game and I woud use this basic skills to write a CGA shader, so I need to limit the colors and set four vec4 to have cyan, magenta, black and white (I would use only this kind of palette and I don’t need dithering)

can you help me??

I’m using only the fragment shader (my game is a 2D game so I need only this)

this is what I’ve written

//
// Simple passthrough fragment shader
//
varying vec2 v_vTexcoord; //this is the coord xy
varying vec4 v_vColour; //this is the color space of the original surface

void main()
{
vec4 color1 = vec4(0.0, 1.0, 1.0, 1.0); //magenta
vec4 color2 = vec4(1.0, 0.0, 1.0, 1.0); //cyan
vec4 color3 = vec4(1.0, 1.0, 1.0, 1.0); //black
vec4 color4 = vec4(0.0, 0.0, 0.0, 1.0); //white

vec4 CGA = vec4(color1,color2,color3,color4); //this doesnt work so well

gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord ); 
//this is a variable that take the original color and draw it as it is

}

thanks a lot

First, you’ll need to explain the problem more clearly.

You want a shader where each pixel is converted to one of those four colours, whichever is closest? Or a shader that clamps to the set of colours which could be obtained by mixing them (i.e. as if you dithered to a CGA palette then filtered the result)? Or what?

In any case, the matrix:


[ 1   1  -1]
[-1   0   1]
[ 0  -1   1]

will transform R,G,B to White,Cyan,Magenta, i.e. [1,1,1]->[1,0,0], [0,1,1]->[0,1,0], [1,0,1]->[0,0,1].

From there, you can clamp or scale to the volume bounded by those colours (i.e. W+C+M<=1), or find which of the colours are nearest (if W+C+M<1/2, it’s black, otherwise it’s whichever component has the largest value).

You can then transform back to R,G,B with


[ 1   0   1]
[ 1   1   0]
[ 1   1   1]

[QUOTE=GClements;1283430]First, you’ll need to explain the problem more clearly.

You want a shader where each pixel is converted to one of those four colours, whichever is closest? Or a shader that clamps to the set of colours which could be obtained by mixing them (i.e. as if you dithered to a CGA palette then filtered the result)? Or what?

[/QUOTE]

I need the first case, I’m trying to convert all the colors in one of those four colors to limit the palette and change all the original colors closest to one of those

I’m a very basic user in shaders, can you write me how to write those informations??

I explane you well, (maybe game maker studio variables are different from usual glsl)

when I create a new shader in game maker studio I have:

//this vertex shader

//
// Simple passthrough vertex shader
//
attribute vec3 in_Position; // (x,y,z)
//attribute vec3 in_Normal; // (x,y,z) unused in this shader.
attribute vec4 in_Colour; // (r,g,b,a)
attribute vec2 in_TextureCoord; // (u,v)

varying vec2 v_vTexcoord;
varying vec4 v_vColour;

void main()
{
vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;

v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;

}

//this fragment shader

//
// Simple passthrough fragment shader
//
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

void main()
{
gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
}

the first two variables (I don’t know what varying it is) are:

v_vTexcoord the texture coordinate xy where the shader is drawn
v_vColour the color rgba

gl_FragColor return the original color of the application surface
gm_BaseTexture i think is a game maker variable

anyway, this is the starting point

how can I write your cga shader here ??

Thank you so much

[QUOTE=Heavybrush;1283431]I need the first case, I’m trying to convert all the colors in one of those four colors to limit the palette and change all the original colors closest to one of those
[/QUOTE]


uniform sampler2D gm_BaseTexture;

varying vec2 v_vTexcoord;
varying vec4 v_vColour;

const mat3 rgb_to_wcm = mat3(1,-1, 0,  1, 0,-1, -1, 1, 1);

void main()
{
    vec4 rgba = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord);
    vec3 wcm = rgb_to_wcm * rgba.rgb;
    vec3 rgb = dot(wcm,vec3(1,1,1)) < 0.5
        ? vec3(0,0,0)
        : wcm.x > wcm.y
            ? (wcm.x > wcm.z ? vec3(1,1,1) : vec3(1,0,1))
            : (wcm.y > wcm.z ? vec3(0,1,1) : vec3(1,0,1));
    gl_FragColor = vec4(rgb, rgba.a);
}

it doesn’t work well
it gives me a compiler error

and as I can see it is more complicated that I could imagine

I really don’t understand why it doesn’t work

anyway thanks a lot

You stand a better chance of getting help if you say what the compiler error is. Those error messages exist for a reason: to assist people in diagnosing the cause of the error.

Reading project file…Error compiling fragment shader:
In Shader shd_CGA at line 2 : ‘gm_BaseTexture’ : redefinition
Compile Failed - Please check the Compile window for any additional information

It doesn’t give additional information

So remove the declaration of that variable.

Game Maker is presumably either adding its own code to the shader or adding its own shader(s) to the program.

ok I leaved the uniform decalration at start
now it works, but is only black and white, how to add also the other two?

now the code is this:


varying vec2 v_vTexcoord;
varying vec4 v_vColour;
 
const mat3 rgb_to_wcm = mat3(1,-1, 0,  1, 0,-1, -1, 1, 1);
 
void main()
{
    vec4 rgba = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
    vec3 wcm = rgb_to_wcm * rgba.rgb;
    vec3 rgb = dot(wcm,vec3(1,1,1)) < 0.5
        ? vec3(0,0,0)
        : wcm.x > wcm.y
            ? (wcm.x > wcm.z ? vec3(1,1,1) : vec3(1,0,1))
            : (wcm.y > wcm.z ? vec3(0,1,1) : vec3(1,0,1));
    gl_FragColor = vec4(rgb, rgba.a);
}

Find a scene with more cyan and/or magenta in it, or bias the comparison against white (e.g. use k*wcm.x instead of wcm.x, for some k less than 1).

Partitioning the RGB colour space according to the closest colour (of black, white, cyan, magenta) results in proportions of 50% black, 20% white, 15% cyan, 15% magenta. Note that cyan and magenta need to be quite saturated to match those primaries rather than black or white. E.g. (1,0.5,1) is on the boundary between white and magenta, while (0.5,1,1) is on the boundary between white and cyan.

thank you so much
the shader works pretty well

I have just another question about it:

how to leave all the alpha information??
(take every pixel that have an alpha < 1 and set it to 0)
(or <= 0.5 and set it to 0 and > 0.5 and set it to 1) //maybe this a better choice, I don’t know

searching for answers I found also this:
https://www.shadertoy.com/view/4dXXzl#

I really like it also if it force the pixel with the dithering that I don’t like so much

I noticed here there is no vertex and fragment separation, so I want to ask you, why??

is it shadertoy a good tool to excercise myself with glsl??
how it works??
what do you think about??

and if you think it is not a good way to learn, what do you use??
how to write shaders and see the result to excercise myself and better study from the manual??

thank you so much

You said you are using GameMaker? Is this 3D or 2D? I don’t think Game Maker does 3D.

So, you probably don’t want a vertex shader. You’re probably intending to do straight pixel shading. (Called a fragment shader in GLSL.) If it’s 3D I can tell you what you need to do. With 2D, not as much.

However, all a fragment/pixel shader does is determine the color of the pixel. The pixel color data probably is coming in as 4 floating point values from 0 to 1. that’s 0% to 100% of each color. Those are RGBA values, which stands for the 3 primary colors that make up all the colors we can see: red, green, and blue. A is for Alpha which is not a color but the transparency, which you probably have figured out.

So, your pixel shader can take whatever data it receives to make decisions on what to do, but it’s output is always just the color of it’s pixel and the alpha value (because blending for transparency happens after this stage and the more transparent it is the more you will see what is behind it). You can alter the alpha value in your shader. So for example, if the texture has an alpha value of .73 for that pixel, you could change anything less than 1.0 to be 0.0. That would change the .73 value to .0 and be fully transparent instead of the potentially 73% transparent in the texture.

Controlling the alpha value in the texture itself requires going into Photoshop, or whatever program you use, and setting it. First, to do that, it has to have an alpha channel, which may or may not be the case.

Yes I use Game Maker which gives me a good and easy way to write shaders with vertex and fragment shader
and yes game maker allows 3D, but I’m focusing on 2D (using 3D in game maker is really possible but also really hard if you are not a good programmer, if you are you can achieve also a good level of realism)

https://www.youtube.com/watch?v=SsjSNThqiY4

I found also shadertoy on the web which gives me an exemple from other people to learn writing shaders

and a tutorial online to use opengl on the web canvas but it’s very complicated

anyway, I make games and for now game maker is a great tool for me, when I will study vertex shader probably I will pass to unity and unreal which I already know how to use, there are also great courses about writing shaders in maya

for what about photoshop I know very well how to change alpha values

my problem is very different:
for sure about sprites that I use inside the game I will use an image editor as photoshop

I’m also going to use particle effects in game maker, and you can set almost everything, but the particles have a birth, life and death
game maker manage them using antialiasing and alpha, it is not mean to be used for old retrostyle pixel games (if you want to give them next gen mechanics)
I cannot use a prerendered particle effects, because it have to randomize

so I have anyway to force the alpha by shader

You can force the alpha to 0 or 1 with e.g.


gl_FragColor = vec4(rgb, rgba.a < 0.5 ? 0.0 : 1.0);

Or you can explicitly discard fragments which would have zero alpha with e.g.:


if (rgba.a < 0.5)
    discard;
gl_FragColor = vec4(rgb, 1.0);

If depth buffering is enabled, the former will still update the depth buffer while the latter won’t. But that probably isn’t an issue for 2D with Game Maker (I’m assuming that it renders from back to front, which is necessary for blending to work correctly).

yes you are right, game maker draw(render) the objects with a z depth from positive to negative

the positive values stay back and the negative values come in front
all the sprites, objects or draw (because you can also draw shapes by code) have an individual alpha

thank you very much for your support
I wish continue studying glsl and for sure I will

If you can advice me to how to learn better and faster it will be very appreciated
I don’t know anything about shaders and I think I will start doing first some exercise from youtube or some courses online, and after I will go deep reading all the manual, I already tried reading all the manual but for me is a little bit complicated, so I think I will start doing some video to have also a visual reference of how things works

maybe there are some courses better than others or more professional, maybe there are some better for vertex shader and other better for fragment shader
if you know something about how to start and how to keep going I will try to take your advice

thank you for all

Hello, I’m sorry to reopen the thread but I need to change this shader
maybe you can help me

I have to change it to swap the palette also if the application surface is not close to it
secondary, I have to render it dynamic (maybe using an uniform variable) because it have to change the palette

and if it possible, can you better comment this code, to let me understand better every step and give an help to this question?

please help me
thank you so much

here the code:


///CGA Shader
varying vec2 v_vTexcoord; //xy
varying vec4 v_vColour; //rgba

#define _ 0.0
#define o (1./3.)
#define b (2./3.)
#define B 1.0

#define check(r,g,b) color=vec4(r,g,b,0.); dist = distance(sample,color); if (dist < bestDistance) {bestDistance = dist; bestColor = color;}

void main(out vec4 v_vColour, in vec2 v_vTexcoord)
{
    float dist;
    float bestDistance = 1000.;
    vec4 color;
    vec4 bestColor;
    
    vec2 pixel = vec2(320,200)-floor( v_vTexcoord.xy / iResolution.xy * vec2(320,200));
    vec4 sample = texture2D(iChannel0, pixel / vec2(320,200));
    
    sample += (texture2D(iChannel1, pixel / iChannelResolution[1].xy)-0.5)*0.5;
    
    // pallete 0
    
    check(_,_,_);
    check(o,B,o);
    check(B,o,o);
    check(B,B,o);
    
    vec4 color1 = bestColor;
    bestDistance = 1000.;
    
    // pallete 1
    
    check(_,_,_);
    check(o,B,B);
    check(B,o,B);
    check(B,B,B);
    
    vec4 color2 = bestColor;
    
    float t = (clamp(sin(iGlobalTime)*4.,-1.,1.)+1.)/2.;
    
    v_vColour = mix(color2, color1, t);
    
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
}

an exemple to how to render dynamic a shader in game maker is this:
here is the shader and how to manipulate it
the shader is a pixel shader and can be manipulated to change the size of the pixels
I have to change the cga colors of the palette


//-----------------------------------------------------------------
//------- PIXELATED SHADER V2--------------------------------------
//-----------------------------------------------------------------

//fragment

varying vec2 v_vTexcoord;
varying vec4 v_vColour;

uniform vec2 size; //the actual size in pixels
uniform float pixelSize;

void main()
{
    vec2 realTexPos = v_vTexcoord * size;
    vec2 floorTexPos = floor(realTexPos / pixelSize) * pixelSize;
    vec2 texPos = floorTexPos / size; //la nuova posizione dei pixel
    gl_FragColor = v_vColour * texture2D( gm_BaseTexture, texPos );
}

this is the code of the object to disable the surface and set the shader:


//object
//create event

///disable application surface
application_surface_draw_enable(false);
pixelSize = 10; //set pixelSize to 10

//draw gui end event

///draw application surface
if(shader_is_compiled(shader)) { 
    shader_set(shader);
    var s = shader_get_uniform(shader,"size"); 
    shader_set_uniform_f(s,surface_get_width(application_surface),surface_get_height(application_surface)); 
    var ps = shader_get_uniform(shader,"pixelSize"); 
    shader_set_uniform_f(ps,pixelSize); 
    draw_surface(application_surface,0,0);
    shader_reset(); 
}
else {
    draw_surface(application_surface,0,0);
}

this is the code to manipulate it:


//step event

///position in base of the mouse
x = mouse_x;
y = mouse_y;
if(mouse_wheel_up()) { 
    pixelSize++; 
}
if(mouse_wheel_down()) { 
    pixelSize = max(1, pixelSize - 1);
}

It doesn’t work well
It gives me a compiler error

yes you are right, i’m sorry
I used the wrong shader

the shader is this:


///CGA Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

const mat3 rgb_to_wcm = mat3(1,-1, 0,  1, 0,-1, -1, 1, 1);

void main()
{
    vec4 rgba = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
    vec3 wcm = rgb_to_wcm * rgba.rgb;
    vec3 rgb = dot(wcm,vec3(1,1,1)) < 0.5
        ? vec3(0,0,0)
        : wcm.x > wcm.y
            ? (wcm.x > wcm.z ? vec3(1,1,1) : vec3(1,0,1))
            : (wcm.y > wcm.z ? vec3(0,1,1) : vec3(1,0,1));
    if (rgba.a < 0.9) discard;
    gl_FragColor = vec4(rgb, rgba.a);
}

This can perhaps to be handled by a transformation of RGBA components via a vec4 to ivec4 reduction/conversion followed by another transformation from this ivec4 to a final vec4 conversion/expansion, with something like this code ?


/// CGA Shader

varying vec4 v_vColour;
varying vec2 v_vTexcoord; 
 
void main()
{
    vec4 srccolor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);

    vec4 reduced = srccolor / 127.0;

    ivec4 converted = reduced;

    ivec4 expanded = converted * 127;
    
    gl_FragColor = expanded;
}

I have not tested this code, so the value of 127 is perhaps not the good value
(I have initially used values of 64 or 85 but CGA colors use only one bit per color component, not 2 bits)

I think that the greaterThan vector function can too be used for to handle the reduction/conversion/expansion steps, like this more little code :


/// CGA Shader (set gl_FragColor to the CGA reduction of v_vColor)

varying vec4 v_vColour;
varying vec2 v_vTexcoord; 
 
void main()
{
    vec4 srcColor = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);

    bvec4 cgaed = greaterThan(srcColor, vec4(0.5f));
    
    gl_FragColor = cgaed;
}

(a threshold color can to be used instead the vec4(0.5f) if you want to handle a different threshold than 0.5f for each RGBA component)