Introduction
This notebook demonstrates a few example uses for pySSV and includes examples for many of the supported features.
In these first few examples we demonstrate some basic fragment shaders.
[ ]:
# Google colab support
try:
# Try enabling custom widgets, this will fail silently if we're not in Google Colab
from google.colab import output
output.enable_custom_widget_manager()
# Install pySSV for this session
%pip install pySSV
except:
pass
[ ]:
import pySSV as ssv
import logging
ssv.ssv_logging.set_severity(logging.INFO)
[ ]:
# Create a new SSVCanvas, the canvas is responsible for managing the OpenGL context, the render widget, and the state of the renderer.
canvas = ssv.canvas()
# Check what graphics adapter we're using
canvas.dbg_log_context()
# Set up a very basic shader program to check it's working
canvas.shader("""
#pragma SSV shadertoy
void mainImage(out vec4 fragColor, in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/iResolution.yy;
// Colour changing over time
vec3 col = sin(uv.xyx + iTime * vec3(3, 4, 5)) * 0.5 + 0.5;
float alpha = smoothstep(0.1, 0.1+2./iResolution.y, 1.-length(uv*2.-1.));
// Output to screen
fragColor = vec4(vec3(col), alpha);
}
""")
[ ]:
# run() starts the render loop, it will continuously render frames until stop() is called or the widget is destroyed.
# We set the stream mode to png here as it supports transparency. In general though, jpg (the default) is much faster.
canvas.run(stream_mode="png")
[ ]:
canvas.stop()
Mouse input
Here’s a basic example of a shader that makes use of mouse position. With the dbg_shader() method, glsl code is generated around your shader to support ShaderToy-like shaders. In this case the canvas resolution is passed is in as iResolution and the mouse position as iMouse.
[ ]:
import pySSV as ssv
canvas1 = ssv.canvas()
canvas1.shader("""
#pragma SSV pixel mainImage
vec4 mainImage( in vec2 fragCoord )
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/uResolution.xx;
float aaScale = 1./uResolution.x;
vec2 mouse = uv-uMouse.xy / uResolution.xx;
// Time varying pixel color
vec3 col = vec3(smoothstep(0.9, .95, 1.-length(mouse)));
col -= 1.-vec3(step(dot(step(abs(mouse), vec2(0.8/uResolution.x, 5./uResolution.x)), vec2(0.5)), 0.5));
col -= 1.-vec3(step(dot(step(abs(mouse), vec2(5./uResolution.x, 0.8/uResolution.x)), vec2(0.5)), 0.5));
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
canvas1.run()
Here’s a more complex shader taken almost directly from ShaderToy.
[ ]:
canvas2 = ssv.canvas()
canvas2.shader("""
#pragma SSV shadertoy
// Copyright Thomas Mathieson all rights reserved
// https://www.shadertoy.com/view/DsffWM
const float motionBlur = 0.3;
const float aa = 0.6;
const vec3 col1 = vec3(13., 45., 140.)/100.;
const vec3 col2 = vec3(255., 20., 50.)/255.;
const vec3 col3 = vec3(21., 191., 112.)/600.;
const vec3 col4 = vec3(0.35, 1., 0.7)*0.65;
const float speed = 0.1;
float sigmoid(float x)
{
return 1.*x/(abs(x)+1.);
}
vec3 sigmoid(vec3 x)
{
return x/(abs(x)+vec3(1.));
}
vec3 saturate(vec3 x)
{
return clamp(x, 0., 1.);
}
vec3 blend(float x, vec3 c)
{
c = pow(c, vec3(x+2.));
return mix(x*c, x*(1.-c), step(x, 0.));
}
float f(vec2 p, float t, vec4 o, vec4 o1, float s, vec4 scale)
{
vec4 i0 = cos(t+o)*vec4(o.xw, o1.xw);
vec4 i1 = sin(t+o1)*vec4(o.xw, o1.xw);
vec4 x0 = i0*s*sin(scale*length(p*o.xy+4.*scale.zw)+o.z+t*o.w);
vec4 x1 = i1*s*sin(scale*length(p*o1.xy)+o1.z+t*o1.w);
return sigmoid(dot(x0+x1, vec4(1.)));
}
vec3 scene(float t, float emphasis, vec2 uv)
{
// "Beautiful" randomness, tuned for aesthetics, not performance
vec2 p = uv * 3.;
t += 160.;
t *= speed;
vec4 scale = vec4(sin(t*vec3(0.25, .5, .75)), cos(t*.95))*.25+.5;
float s0 = f(p, t, vec4(6.,9.,2.,1.5), vec4(2.,9.,7.,3.), .25, scale);
float s1 = f(p, t, vec4(2.,6.5,1.5,4.0), vec4(3.,2.5,3.8,1.6), .5, scale);
float s2 = sigmoid(s0/s1)*0.5;
float s3 = f(p, t, vec4(2.,9.,7.,3.), vec4(6.,3.,2.,1.5), .125, scale);
float s6 = f(p*1.5, t, vec4(6.,4.,8.,2.5), vec4(3.2,1.6,9.7,7.9), .25, scale);
float s7 = f(p*1.3, t, vec4(2.,6.5,1.5,4.0), vec4(3.,2.5,3.8,1.6), .5, scale);
float s8 = sigmoid(s6/s7+s0)*0.7;
vec3 c = vec3(sigmoid((blend(s8,col1)+blend(s2,col2)+blend(s1,col3)+s7*1.)*1.1)*.7+.5);
float grad = sigmoid(pow(length(uv*2.-1.)+s3*.3, 5.))*1.5;
float accent = 1.-sigmoid((pow(2.5, abs(sigmoid(s8+s0+s1))-1.)-.45-(emphasis*0.1))*1000./(1.+30.*grad+20.*emphasis));
c = mix(c, c.r*.3+col4*.8, accent);
return clamp(vec3(c), 0., 1.);
}
void mainImage(out vec4 fragColor, in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/iResolution.xx;
float aaScale = 1./iResolution.x;
vec2 mouse = uv-iMouse.xy /iResolution.xx;
float emp = sigmoid(1./pow(length(mouse*1.), 1.8)*.02);
// Time varying pixel color
vec3 col = scene(iTime, emp, uv);
//col += scene(iTime + motionBlur*0.001, emp, uv + aaScale*aa*vec2(0.,1.))
// + scene(iTime + motionBlur*0.002, emp, uv + aaScale*aa*vec2(1.,0.));
//col /= 3.;
// Output to screen
fragColor = vec4(vec3(col), 1.0);
}
""")
canvas2.run(stream_quality=100)
Shader Templates
pySSV makes use of a shader templating system to reduce boilerplate. Many shader templates are provided but you can of course write your own (instructions for which are in the documentation). Shader templates can be just a thin layer glsl boilerplate or can contain significant a amounts of high level functionality as shown in the example below which uses the sdf template for signed distance field rendering. This template takes a distance function as an entrypoint and generates the renderer
code.
Shader templates are specified using the #pragma SSV <template_name> [template arguments...] directive. Arguments are defined in the shader template and are specified similar to command line arguments (they are parsed by python’s argparse module internally).
[ ]:
import pySSV as ssv
canvas3 = ssv.canvas()
canvas3.shader("""
#pragma SSV sdf sdf_main --camera_distance 2. --rotate_speed 1.5 --render_mode SOLID
// SDF taken from: https://iquilezles.org/articles/distfunctions/
float sdCappedTorus(vec3 p, vec2 sc, float ra, float rb) {
p.x = abs(p.x);
float k = (sc.y*p.x>sc.x*p.y) ? dot(p.xy,sc) : length(p.xy);
return sqrt( dot(p,p) + ra*ra - 2.0*ra*k ) - rb;
}
float sdf_main(vec3 p) {
float t = 2.*(sin(uTime)*0.5+0.5)+0.2;
return sdCappedTorus(p, vec2(sin(t), cos(t)), 0.5, 0.2);
}
""")
canvas3.run(stream_quality=100)
Multi-Pass Rendering
pySSV provides support for multi-pass rendering and multiple draw calls within a pass.
The rendering system renders draw calls belonging to render buffers belonging to canvases (SSVCanvas -owns-> SSVRenderBuffer -owns-> SSVVertexBuffer). When you create an SSVCanvas, internally, it creates an SSVRenderBuffer which itself creates an SSVVertexBuffer to draw into. canvas.shader() is actually shorthand for canvas.main_render_buffer.full_screen_vertex_buffer.shader() since a shader must belong to an individual draw call (since vertex buffers and draw
calls are linked 1-1, we use the terms interchangably here).
To start creating your own render buffers and draw calls use the following API:
[ ]:
import pySSV as ssv
import numpy as np
canvas4 = ssv.canvas(use_renderdoc=True)
# Create a new render buffer on this canvas
rb = canvas4.render_buffer(size=(640, 480), name="renderBuffer1")
# Now we can render full-screen shaders on both the main render buffer and our new render buffer
### Draw diagonal stripes in the background, and composite renderBuffer1 on top
canvas4.shader("""
#pragma SSV pixel mainImage
vec4 mainImage(in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/uResolution.xy;
// Some diagonal stripes for the background
vec3 col = vec3(step(fract((fragCoord.x+fragCoord.y+uTime*50.)/50.0), 0.5));
// Now blend the output of renderBuffer1 on top
vec4 rb1 = texture(renderBuffer1, uv);
col = mix(col, rb1.rgb, rb1.a);
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
### Draw a circle wwith a colour changing gradient in renderBuffer1
rb.shader("#pragma SSV render_test")
# If we wanted to add another draw call to our new render buffer we would use the vertex_buffer() method
vb = rb.vertex_buffer()
# Now we can populate this vertex buffer
vb.update_vertex_buffer(np.array([
# X Y R G B
-1.0, -1.0, 1.0, 0.0, 0.0,
1.0, -1.0, 0.0, 1.0, 0.0,
0.0, 1.0, 0.0, 0.0, 1.0], # This should make a single triangle
dtype='f4',
))
# And assign a shader to it's draw call
### Draw a colourful triangle on top of renderBuffer1
vb.shader("""
#pragma SSV pixel mainImage
vec4 mainImage(in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/uResolution.xy;
// Color from the vertex colours
vec3 col = color.rgb;
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
canvas4.run(stream_quality=100)
Camera
When you create a canvas, an SSVCamera is also created and attached to it. This automatically recevies input from the canvas and makes a view and projection matrix available in the shader to be used by shaders requiring 3d perspective transformations.
[ ]:
import pySSV as ssv
import numpy as np
canvas5 = ssv.canvas()
canvas5.main_render_buffer.full_screen_vertex_buffer.update_vertex_buffer(np.array([
# X Y R G B
-1.0, -1.0, 1.0, 0.0, 0.0,
1.0, -1.0, 0.0, 1.0, 0.0,
0.0, 1.0, 0.0, 0.0, 1.0], # This should make a single triangle
dtype='f4',
))
canvas5.shader(
"""
#pragma SSV vert mainVert
VertexOutput mainVert()
{
VertexOutput o;
vec4 pos = vec4(in_vert, 1., 1.0);
pos = uViewMat * pos;
pos = uProjMat * pos;
o.position = pos;
o.color = vec4(in_color, 1.);
return o;
}
""")
# We can configure the camera settings using the `main_camera` field of the canvas
canvas5.main_camera.fov = 60
canvas5.run(stream_quality=100)
Data Input And Custom Textures
There are 3 ways of getting data into a shader for rendering:
As vertex data
As a uniform
As a texture
Uniforms
Uniforms are the simplest to use, and we’ve already seen them in previous examples. Uniforms are great for small amounts of data which you might want to change frequently such as lighting paramaters, camera transformations, etc… There is a limit to the number of uniforms you can declare and the total amount of memory they consume, this limit depends on your platform but is generally big enough that you don’t need to consider it; that is as long as you don’t declare large (>1000 elements) arrays as uniforms.
To assign values from python to a uniform it must be in a compatible type. Uniforms generally only accept numeric data (ints and floats) as scalars, vectors (up to 4 components), or matrices (up to 4x4 components); any appropriately sized array-like python type can be assigned to a uniform vector. See https://www.khronos.org/opengl/wiki/Data_Type_(GLSL) for more details on supported GLSL types to use with uniforms.
[ ]:
import pySSV as ssv
canvas5 = ssv.canvas()
canvas5.shader("""
#pragma SSV pixel mainImage
uniform vec3 customColour;
vec4 mainImage(in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/uResolution.xy;
// Color from the uniform
// Some diagonal stripes
vec3 col = vec3(step(fract((fragCoord.x+fragCoord.y+uTime*50.)/50.0), 0.5));
// Now colour them with the uniform
col *= customColour;
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
canvas5.update_uniform("customColour", (1, 0, 1)) # Magenta
canvas5.run(stream_quality=100)
Vertex Data
For sparse spatial data, the vertex buffer is a good method of getting data into the shader.
Triangle meshes and point clouds are natural choices data which can use the vertex buffer. Vertices can contain any number of attributes (limited by the graphics backend) in addition to position, such as colour, normals, etc… Vertex data is processed in parallel in the vertex shader which is a good place to put any per-vertex transformations. The outputs of the vertex shader are then interpolated automatically into the fragment (pixel) shader.
You can define your own vertex data structures and specify how vertex attributes are bound in the shader using the vertex_attributes parameter in the update_vertex_buffer() method. To take full advantage of custom vertex_attribtues though you’ll need to write your own shader template.
In the following example we generate a point cloud with the shape (64, 64, 6) (which is later flattened into 1D array). The vertex attributes in this example are vec3 in_vert; vec3 in_color this maps onto the 6 components in the point cloud. The point_cloud shader template used here, exposes an entrypoint in the vertex stage to perform transformations on the points themselves (in our case we perform the perspective transformation for the camera). The template then passes this to a
geometry shader, which generates sprites (2-triangle primitives) for each vertex (use the dbg_preprocess_shader() described below to see the final shader generated by the template preprocessor to see how this works).
[ ]:
# Generate some points
def generate_points():
width, depth = 64, 64
scale = 3
v_scale = 0.5
f = 0.01
verts = np.zeros((width, depth, 6), dtype='f4')
for z in range(depth):
for x in range(width):
dx = width/2 - x
dz = depth/2 - z
y = np.sin((dx*dx+dz*dz)*f) * v_scale
verts[z, x, :3] = [x/width * scale, y, z/depth * scale]
verts[z, x, 3:6] = [y/v_scale, abs(y/v_scale), np.sin(y/v_scale*10.)*0.5+0.5]
return verts.flatten()
[ ]:
import pySSV as ssv
import numpy as np
canvas5 = ssv.canvas(use_renderdoc=True)
# Set the contents of default vertex buffer on the main pass (normally used for full-screen shaders, but in this case hijacked for this example)
canvas5.main_render_buffer.full_screen_vertex_buffer.update_vertex_buffer(generate_points())
canvas5.main_camera.target_pos = np.array((1.5, 0, 1.5))
canvas5.shader("""
#pragma SSV point_cloud mainPoint
VertexOutput mainPoint()
{
VertexOutput o;
vec4 pos = vec4(in_vert, 1.0);
pos = uViewMat * pos;
pos = uProjMat * pos;
o.position = pos;
o.color = vec4(in_color, 1.);
o.size = 30.0/uResolution.x;
return o;
}
""")
canvas5.run(stream_quality=100)
Textures
To get large amounts of data into a shader textures are ideal.
Most GPUs support 2D and 3D textures in a variety of formats (usually from 8, 16, and 32 bits per component, as floats of uints, up to 4 components per pixel (RGBA)); pySSV automatically attempts to determine the correct texture dimensions and format from a NumPy array. Different GPUs have different limitations as to the maximum dimensions (width, height, and depth) for textures which cannot be exceeded, for 2D textures 16384x16384 is usually limit, for 3D textures this is often smaller. That being said a 16384x16384 texture with 4x32 bit components per pixel represents 4 GB of memory.
Textures have a few useful features which can be exploited pretty much for free:
Texture interpolation: textures can be sampled using nearest neighbour interpolation or bilinear / trilinear (for mipmaps or 3d textures), interpolation.
Texture repetition: when sampling textures outside of the usual 0-1 range, they can be set to either repeat or clamp to edge values.
Texture mipmaps: (only recommended for textures with power-of-2 dimensions) the GPU can efficiently generate image pyramids for textures which can be sampled with linear interpolation between levels in the shader. These can be used to approximate blurring operations, compute averages, or to prevent texture aliasing.
[ ]:
import pySSV as ssv
import numpy as np
canvas5 = ssv.canvas(use_renderdoc=True)
# Here we generate a simple 3x3 single component texture. By default, pySSV will attempt to treat this as a 3x1
# texture with 3 components (since the height is less than the maximum number of components in a texture (4)), as
# such we use the `force_2d` parameter to tell pySSV to treat the 2nd dimension of the array as height instead of
# components.
texture = canvas5.texture(np.array([
[0., 0.1, 0.2],
[0.3, 0.4, 0.5],
[0.6, 0.7, 0.8]
], dtype=np.float16), "uTexture1", force_2d=True)
# texture.linear_filtering = False
texture.repeat_x, texture.repeat_y = False, False
texture.linear_filtering = False
canvas5.shader("""
#pragma SSV pixel mainImage
// Provided that the texture is declared on the canvas *before* the shader is, then it's uniform will be
// automatically declared in the shader by the preprocessor.
//uniform sampler2D uTexture1;
vec4 mainImage(in vec2 fragCoord)
{
// Normalized pixel coordinates (from 0 t 1)
vec2 uv = fragCoord/uResolution.xy;
// Color from the uniform
// Some diagonal stripes
vec3 col = vec3(step(fract((fragCoord.x+fragCoord.y+uTime*50.)/50.0), 0.5));
// Now colour them with the texture
col *= vec3(texture(uTexture1, uv).r);
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
canvas5.run()
[ ]:
if "canvas" in globals():
canvas.stop()
if "canvas1" in globals():
canvas1.stop()
if "canvas2" in globals():
canvas2.stop()
if "canvas3" in globals():
canvas3.stop()
if "canvas4" in globals():
canvas4.stop()
if "canvas5" in globals():
canvas5.stop()
if "canvas6" in globals():
canvas6.stop()
Debugging Shaders
Shaders can get quite complex so pySSV provides a few tools to simplify debugging your shaders.
Preprocessor Dump
It can be helpful to view the GLSL generated by the pre processor to understand why things are going wrong:
[ ]:
import pySSV as ssv
canvas1 = ssv.canvas()
shader = canvas1.dbg_preprocess_shader("""
#pragma SSV pixel mainImage
vec4 mainImage( in vec2 fragCoord )
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/uResolution.xx;
float aaScale = 1./uResolution.x;
vec2 mouse = uv-uMouse.xy / uResolution.xx;
// Time varying pixel color
vec3 col = vec3(smoothstep(0.9, .95, 1.-length(mouse)));
col -= 1.-vec3(step(dot(step(abs(mouse), vec2(0.8/uResolution.x, 5./uResolution.x)), vec2(0.5)), 0.5));
col -= 1.-vec3(step(dot(step(abs(mouse), vec2(5./uResolution.x, 0.8/uResolution.x)), vec2(0.5)), 0.5));
// Output to screen
return vec4(vec3(col), 1.0);
}
""")
print(shader)
List Shader Templates
You can also get a list of all the installed shader templates.
[ ]:
print(canvas1.dbg_query_shader_templates(additional_template_directory=None))
Query Shader Template Arguments
And you can query a shader template for it’s arguments.
[ ]:
print(canvas1.dbg_query_shader_template("sdf"))
OpenGL Context
If you’re trying to track down a driver bug or platform specific oddity, having the graphics adapter information can be helpful
[ ]:
canvas1.dbg_log_context(full=True)
Frame Times
pySSV also provides rudimentry frame time logging to identify bottlenecks.
[ ]:
canvas1.dbg_log_frame_times(enabled=True)
# Then you just need to run the canvas
# canvas1.dbg_render_test()
# canvas1.run()
[ ]:
canvas1.stop(force=True)
Debugging Shaders With Renderdoc
Renderdoc is a powerful open-source graphics debugger. To use Renderdoc with pySSV, install the python bindings for the Renderdoc in-app API (https://github.com/space928/pyRenderdocApp/):
[ ]:
%pip install pyRenderdocApp
Then simply create a new SSVCanvas with use_renderdoc_api set to True and the Renderdoc API will be loaded automatically. To capture a frame simply press the Renderdoc logo button in the widget.
[ ]:
try:
import pySSV as ssv
import pyRenderdocApp
canvas = ssv.canvas(use_renderdoc=True)
canvas.shader("#pragma SSV render_test")
canvas.run()
except ImportError:
print("Couldn't import pyRenderdocApp!")
[ ]:
canvas.stop(force=True)
[ ]: