Shaders – Gaussian transformation matrix mission to a 2D viewspace transformation matrix

Basically, I wish to mission:
A 3D transformation matrix of a Gaussian (or on this case by simplicity a sphere of the unit) that features scale, rotation and translation on this planet house TO
A 2D transformation matrix for an ellipse of the unit within the display house

I’m attempting to symbolize the Gaussians effectively throughout the limitations of the API of a recreation. It’s a unit recreation, and every thing I actually have is Meshes and Shaders.

How do I do that?

I additionally surprise if this could possibly be a query for arithmetic.

Presently, after caressing the chatgpt for some time, I managed to spit some fragments that I managed to rebuild, virtually works. It really works completely with distance projection or perspective, however the perspective has an vital error carefully. Right here is my code:

float2x2 Project3DGaussianTo2D(
    float3x3 gaussianLocalTransform,
    float3x3 cameraRotationWorldToCamera,
    float3 cameraRelativeDirection
) {
    // Step 1: 3D Covariance Matrix
    float3x3 G = gaussianLocalTransform;
    float3x3 C = mul(G, transpose(G));

    // Step 2: Remodel covariance to digital camera/view house
    float3x3 R = cameraRotationWorldToCamera;
    float3x3 C_view = mul(R, mul(C, transpose(R)));

    // Step 3: Projection Jacobian J
    float3 heart = mul(R, cameraRelativeDirection);
    float x = heart.x;
    float y = heart.y;
    float z = heart.z;
    float invZ = 1.0 / z;
    float invZ2 = invZ * invZ;

    // Corrected Jacobian utilizing perspective projection ∂proj/∂pos
    float2x3 J;
    J(0) = float3(invZ, 0, -x * invZ2);
    J(1) = float3(0, invZ, -y * invZ2);

    // Step 4: Challenge 3D covariance to 2D
    float scaleZ = 1.0 / (z * z);
    float2x2 C_2D = mul(J, mul(C_view * scaleZ, transpose(J)));

    // Step 5: Eigen-decomposition to get ellipse axes
    float a = C_2D(0)(0), b = C_2D(0)(1);
    float c = C_2D(1)(0), d = C_2D(1)(1);
    float T = a + d;
    float D = a * d - b * c;
    float disc = sqrt(max(T*T - 4*D, 0));
    float lambda1 = 0.5 * (T + disc);
    float lambda2 = 0.5 * (T - disc);

    // Deal with eigenvectors
    float2 majorAxis = normalize(float2(b, lambda1 - a));
    if (dot(majorAxis, majorAxis) 

I actually do not know the way every thing works if I am trustworthy. I simply want this magical sauce to work.

And rendered within the unit utilizing a flat mesh with a airplane of UV coincidence, decreased to zero within the blender and sporting a shador:

v2f vert(appdata v) {
    v2f o;
    float2 uv = float2(saturate(v.uv.x), v.uv.y);
    uv = uv - 0.5;
    o.uv = uv*2;
    
    float3x3 sphereMatRot = ...;
    float3x3 sphereMatScale = ...;
    
    float3 cameraTangentX = mul(UNITY_MATRIX_V(0).xyz, transpose((float3x3)unity_WorldToObject));
    float3 cameraTangentY = mul(UNITY_MATRIX_V(1).xyz, transpose((float3x3)unity_WorldToObject));
    float3 cameraDir = normalize(mul(unity_WorldToObject, float4(_WorldSpaceCameraPos.xyz, 1)) - v.vertex);
    float3x3 cameraRot = ExtractRotation(UNITY_MATRIX_V);
    
    // ===== PROJECT SPHERE TO ECLIPSE
    float2x2 ellipseTrans = Project3DGaussianTo2D(
        mul(sphereMatRot, sphereMatScale),
        cameraRot,
        cameraDir
    );
    uv = mul(ellipseTrans, uv.xy);
    
    float3 offset = uv.x * cameraTangentX + uv.y * cameraTangentY;
    o.pos = UnityObjectToClipPos(v.vertex + float4(offset, 0));
    return o;
}
float4 frag(v2f i) : SV_Target {
    float circle = saturate(1-length(i.uv));
    if (circle ```