-
Notifications
You must be signed in to change notification settings - Fork 27
How to use and write mesh loaders, best practicies
Well, before we start make sure you have knowledge what an Asset is. Get involved if you don't know anything about it: https://github.com/buildaworldnet/IrrlichtBAW/wiki/Asset-Pipeline-Philosophy.
There are two ways of creating and preparing an Asset. You can create it by your own or use one of predefined loaders for that purpose. In case of using loaders, a few things have to be performed to get the Asset.
A user can programmatically/procedurally (without loading any assets) create all of these, like in the example 03.GPUMesh, but loaders are provided for flexibility. Let's take a look for loaders that creates and returns ICPUMesh, ICPUMeshBuffer or ICPURenderpassIndependentPipeline - they must construct (or return cached) pipeline objects with reasonable default shaders attached. But it is all about ICPU objects with mutable state - the reason is following. The user will have their own rendering technique in mind (such as deferred rendering for instance) and will "more often than not" replace the pipeline entirely (maybe while keeping some of the initial state in the one that will replace the original) and when all is done and prepared, such an ICPU Asset will be converted to inmutable IGPU object.
To get started you have to consider there is a device you can use to get a video driver, scene manager and Asset manager. A device is a properly created irr::core::smart_refctd_ptr<irr::IrrlichtDevice>
auto* driver = device->getVideoDriver();
auto* smgr = device->getSceneManager();
auto* am = device->getAssetManager();Next step is to provided parameters that will be used for loading purpose to specify and force some options used for loading process.
asset::IAssetLoader::SAssetLoadParams loadingParams;Having done it, you can execute a functions returning Assets. Note we didn't specified any options in loading parameters, so default options will be in use. Remember that Assets are always loaded as a bundle (set of same type Assets), not as a single Asset!
auto meshes_bundle = am->getAsset("../../media/sponza/sponza.obj", loadingParams);It may occur the loaded Asset bundle is empty, so you should provide a function checking whether there are actually Assets in bundle. For tutorial purpose, I will assert if it's empty, but it isn't necessary though.
assert(!meshes_bundle.isEmpty());Now we should pull our Asset knowing where it is placed in bundle. For loaders forced to load always a single Asset you can get it from beginning of bundle. Pay attention we know from loaded content it's a mesh Asset - ICPUMesh. It's important to know the type of the Asset, because you will have to cast your Asset to use it properly. If you cast it wrong, your program will fall apart.
auto mesh = meshes_bundle.getContents().first[0];
auto mesh_raw = static_cast<asset::ICPUMesh*>(mesh.get());Instead of casting it explicitly, you should use
IAsset::castDown<asset::ICPUMesh>(mesh);and we are done.
Before you will be able to start handling data in your loader, you have to at least override some functions derived from IAssetLoader.
virtual bool isALoadableFileFormat(io::IReadFile* _file) const = 0;virtual const char** getAssociatedFileExtensions() const = 0;virtual SAssetBundle loadAsset(io::IReadFile* _file, const SAssetLoadParams& _params, IAssetLoaderOverride* _override = nullptr, uint32_t _hierarchyLevel = 0u) = 0;
So you need to check signature of file in isALoadableFileFormat, because the manger can exclude a file that is invalid without reading whole data included in it. You also need to put available extensions a loader can handle in getAssociatedFileExtensions. Finally, you should define loadAsset function where you will contain all stuff such as private functions encapsulating your implementation of loading data and handle loading at last.
Open an any loader to see how those functions may look like in it's implementation.
If you ever had to use member variables while loading - store them in a predefined struct called SContext. It's due to multithreading purposes.
It is a class for user to override functions to facilitate changing the way assets are loaded. You can use it to get a file name though, but important is that many behaviours may be overridden.
You can use it for pulling dependent Assets like textures for a model by specifing hierarchy level. For more information about hierarchy levels, read Irrlicht's documentation.
Look at COBJMeshFileLoader that loads textures from materials specified in an .mtl file. You should have 3+ loaders for it:
- OBJ loader that only reads .obj produces an asset::ICPUMesh necessarily invokes an MTL loader
-
MTL loader that only reads .mtl files and produces (or returns) a half-filled
asset::ICPURenderpassIndependentPipeline(that OBJ can modify slightly for raster and vertex input parameters), naturally it invokes image loaders for needed textures. - Texture loaders for PNG, JPEG, etc.
If something cannot be passed within the returned asset bundle, use metadata to communicate between loaders
COBJMeshFileLoader mentioned earlier will probably want to know the material parameters that go with an MTL pipeline, but a pipeline does not include the actual parameters used such as push constants or descriptor sets only the layouts (generic material definition, not specific).
So the specific material definitions should come in the MTL-specific-metadata-derived-class that COBJMeshFileLoader could use and can be fetched from it for instance in hot-loop to fill your Uniform Buffer Object though which you will update and send the data to the GPU with.
You can over-declare descriptor set layouts, i.e. a set that has every single resource and more that a shader could use (all the textures, all the buffers for all shader types).
It's completely okay to have a descriptor in a set that is not used by the shader (not the other way round), and it's okay not to populate a descriptor set binding that was specified in the layout as long as it's not statically used by the shader (dead code).
This helps the engine do less strenous resource rebinds (see Vulkan pipeline compatibility rules).
Most formats such as STL, PLY, OBJ and even glTF declare a very limited set of materials.
You will find that instead of creating a new ICPUSpecializedShader for every single new ICPURenderpassIndependentPipeline, it would make much more sense to create the few ICPUSpecializedShaders and even ICPUDescriptorSetLayout as well as pipeline layouts in the constructor of the loader and cache them under a special path (similar to GLSL built-in includes) in the asset manager cache (then remove upon loader destruction).
You should be aiming to use as few ICPUShader as possible, and have them compile to SPIR-V, there isn't really a reason to not generate all the ICPUShaders you are going to use in the constructor of the pipeline loader.
is only when specialization constants cannot achieve the material, examples:
- different shader in-out blocks (vertex inputs, shader in-out blocks, fragment color outputs)
- types of variables/descriptors or declarations (especially structs) need to change when you have a reason to save on descriptor set bindings (2 and 3 apply to descriptor set layouts)
Normally you can declare all the bindings you'll need for all descriptors, then if specialization constant disables their use, they can remain both in the unspecified in the layout and unpopulated in the set.
Note: when you have different vertex inputs or fragment outputs, you need a separate shader only when the shader-format changes, not the API. So you can still use the same shader if you change from EF_R32G32_SFLOAT to EF_R8G8_UNORM, because the input/output on the shader side is still a vec2. However if you change between SFLOAT/UNORM/SNORM and INT and UINT, then you'd have a problem because declaration goes from vecN to ivecN to uvecN in the GLSL.
When you can pass the argument to an if or switch as a single push_constant or UBO (bit extraction allowed).
Note that switching between materials implemented via uber-shader does not cause unused descriptors to not be statically-used unlike implementing it via separate shader specializations. So the descriptor in the layout will have to be populated if only a flow-control-statement dependent on a push constant or UBO guards its usage. This might be undesireable especially for large buffers or textures as they might waste GPU cache, so sometimes specializing via separate specializations or even ICPUShaders is acceptable.
- STL normal attribute plus optional color attribute so it would be logical to only create
- two vertex
- fragment shader pair
- descriptor layout
- pipeline layout (one with per-vertex color, one without)
Even the pipeline you'd be returning could only come in two variants (not many parameters could change).
-
OBJ only defines 10 illumination models, some of which we can't support out of the box (raytracing and similar) so there will probably only be a few permutations. You could collapse some of the permutations through the use of an uber-shader, the rest could be dealt with specialization constants. However OBJ vertex attributes may change (mostly presence, format irrelevant) so a few separate shaders may be needed. All in all, all the shaders and possibly even specializations + layouts can be precomputed for MTL/OBJ.
-
BAW format allows to serialize everything without limits, so nothing can be precomputed.
We want to avoid redefinitions of Frensel, GGX, Smith, etc. functions that we've already developed (for the BRDF explorer and others). Use the built-in headers!
If you need some functions or formulae, then make a built-in GLSL header with them and check with @devshgraphicsprogramming if it can be added to the engine core (outside and before the loader, GLSL code shared with all), we want as many GLSL headers in the engine core as possible.
Even if it can't go into engine core, split out the common parts of your shaders into built-in includes, and register the built-in includes with the engine in the AssetLoader's constructor!
You could do
#include "whatever/path/we/have/for/builtins/DefaultCameraData.glsl"to include some predefined structures, for instance
struct DefaultCameraData
{
mat4x3 view;
mat4x3 viewInverse;
mat4 proj;
mat4 projInverse;
mat4 projView;
mat4 projViewInverse;
};The system provides also various scenarios, for example when a specific location is needed in layout qualifier
#include "whatever/path/we/have/for/builtins/defaultCameraUBO/location/${loc}.glsl"
#include "whatever/path/we/have/for/builtins/DefaultCameraData.glsl"
layout(set=1, location=${loc}, row_major) DefaultPerViewUBO
{
DefaultCameraData data;
} defaultPerViewUBO;You want to guard every single function definition and descriptor set declaration with something akin to a header guard.
#ifndef _FRAG_OUTPUT_DEFINED_
#define _FRAG_OUTPUT_DEFINED_
layout(location=0) out vec4 color;
#endif
#ifndef _FRAG_MAIN_DEFINED_
#define _FRAG_MAIN_DEFINED_
void main()
{
BSDFIsotropicParams parameters;
... // generate parameters from lightPos, cameraPos and tangent-space
Spectrum output = bsdf_eval(parameters);
color = output.value;
}
#endifThen by prepending the appropriate forward declaration with the same guard, and appending a definition, the programmer could override your shader's functions, just like this.
Append:
#define _FRAG_OUTPUT_DEFINED_
layout(location=0) out uint color;
#define _FRAG_MAIN_DEFINED_
void main();Postpend:
void main()
{
color = packUnorm4x8(vec4(1.0,0.0,0.0,0.0));
}
#ifndef _BSDF_DEFINED_
#define _BSDF_DEFINED_
// provides `BSDFIsotropicParams`, `BSDFAnisotropicParams`, `BSDFSample` and associated functions like `calc
#include "irr/builtin/glsl/bsdf/common.h"
// Spectrum can be exchanged to a float for monochrome
#define Spectrum vec3
//! This is the function that evaluates the BSDF for specific view and observer direction
// params can be either BSDFIsotropicParams or BSDFAnisotropicParams
Spectrum bsdf_cos_eval(in BSDFIsotropicParams params)
{
...
}
//! generates a incoming light sample position given a random number in [0,1]^2 + returns a probability of the sample being generated (ideally distributed exactly according to bsdf_cos_eval)
BSDFSample bsdf_cos_gen_sample(in vec2 _sample)
{
...
}
//! returns bsdf_cos_eval/Probability_of(bsdf_cos_gen_sample) but without numerical issues
Spectrum bsdf_cos_sample_eval(out vec3 L, in ViewSurfaceInteraction interaction, in vec2 _sample)
{
// sample evil implementation (0 div 0 possible and bad efficiency)
BSDFSample _sample = bsdf_cos_gen_sample(smpl);
L = _sample.L;
return bsdf_cos_eval(calcBSDFIsotropicParams(interaction,_sample.L))/_sample.probability;
}
#endifIt should have BRDF Params as following for instance
// do not use this struct in SSBO or UBO, its wasteful on memory
struct DirAndDifferential
{
vec3 dir;
// differentials at origin, I'd much prefer them to be differentials of barycentrics instead of position in the future
mat3x2 dPosdScreen;
};
// do not use this struct in SSBO or UBO, its wasteful on memory
struct ViewSurfaceInteraction
{
DirAndDifferential V; // outgoing direction, NOT NORMALIZED; V.dir can have undef value for lambertian BSDF
vec3 N; // surface normal, NOT NORMALIZED
};
// do not use this struct in SSBO or UBO, its wasteful on memory
struct BSDFSample
{
vec3 L; // incoming direction, normalized
float probability; // for a single sample (don't care about number drawn)
};
// do not use this struct in SSBO or UBO, its wasteful on memory
struct BSDFIsotropicParams
{
float NdotL;
float NdotL_squared;
float NdotV;
float NdotV_squared;
float VdotL; // same as LdotV
float NdotH;
float VdotH; // same as LdotH
// left over for anisotropic calc and BSDF that want to implement fast bump mapping
float LplusV_rcpLen;
// basically metadata
vec3 L;
ViewSurfaceInteraction interaction;
};
// do not use this struct in SSBO or UBO, its wasteful on memory
struct BSDFAnisotropicParams
{
BSDFIsotropicParams isotropic;
float TdotL;
float TdotV;
float TdotH;
float BdotL;
float BdotV;
float BdotH;
// useless metadata
vec3 T;
vec3 B;
};and Parameter Getters. Note that only most performant getter functions (with identities) are provided (in a core API engine built-in).
// chain rule on various functions (usually vertex attributes and barycentrics)
vec2 applyScreenSpaceChainRule1D3(in vec3 dFdG, in mat3x2 dGdScreen)
{
return dFdG*dGdScreen;
}
mat2 applyScreenSpaceChainRule2D3(in mat2x3 dFdG, in mat2 dGdScreen)
{
return dFdG*dGdScreen;
}
mat3x2 applyScreenSpaceChainRule3D3(in mat3x3 dFdG, in mat3x2 dGdScreen)
{
return dFdG*dGdScreen;
}
mat4x2 applyScreenSpaceChainRule4D3(in mat4x3 dFdG, in mat3x2 dGdScreen)
{
return dFdG*dGdScreen;
}
// only in the fragment shader we have access to implicit derivatives
ViewSurfaceInteraction calcFragmentShaderSurfaceInteraction(in vec3 CamPos, in vec3 SurfacePos, in vec3 Normal)
{
ViewSurfaceInteraction interaction;
interaction.V.dir = CamPos-SurfacePos;
interaction.V.dPosdScreen[0] = dFdx(SurfacePos);
interaction.V.dPosdScreen[1] = dFdy(SurfacePos);
interaction.N = Normal;
return interaction;
}
// when you know the projected positions of your triangles (TODO: should probably make a function like this that also computes barycentrics)
ViewSurfaceInteraction calcBarycentricSurfaceInteraction(in vec3 CamPos, in vec3 SurfacePos[3], in vec3 Normal[3], in float Barycentrics[2], in vec2 ProjectedPos[3])
{
ViewSurfaceInteraction interaction;
// Barycentric interpolation = b0*attr0+b1*attr1+attr2*(1-b0-b1)
vec3 b = vec3(Barycentrics[0],Barycentrics[1],1.0-Barycentrics[0]-Barycentrics[1]);
mat3 vertexAttrMatrix = mat3(SurfacePos[0],SurfacePos[1],SurfacePos[2]);
interaction.V.dir = CamPos-vertexAttrMatrix*b;
// Schied's derivation - modified
vec2 to2 = ProjectedPos[2]-ProjectedPos[1];
vec2 to0 = ProjectedPos[0]-ProjectedPos[1];
float d = 1.0/determinant(mat2(to2,to1)); // TODO double check all this
mat3x2 dBaryd = mat3x2(vec3(v[1].y-v[2].y,to2.y,to0.y)*d,-vec3(v[1].x-v[2].x,to2.x,t0.x)*d);
//
interaction.dPosdScreen = applyScreenSpaceChainRule3D3(vertexAttrMatrix,dBaryd);
vertexAttrMatrix = mat3(Normal[0],Normal[1],Normal[2]);
interaction.N = vertexAttrMatrix*b;
return interaction;
}
// when you know the ray and triangle it hits
ViewSurfaceInteraction calcRaySurfaceInteraction(in DirAndDifferential rayTowardsSurface, in vec3 SurfacePos[3], in vec3 Normal[3], in float Barycentrics[2])
{
ViewSurfaceInteraction interaction;
// flip ray
interaction.V.dir = -rayTowardsSurface.dir;
// do some hardcore shizz to transform a differential at origin into a differential at surface
// also in barycentrics preferably (turn world pos diff into bary diff with applyScreenSpaceChainRule3D3)
interaction.V.dPosdx = TODO;
interaction.V.dPosdy = TODO;
vertexAttrMatrix = mat3(Normal[0],Normal[1],Normal[2]);
interaction.N = vertexAttrMatrix*b;
return interaction;
}
// will normalize all the vectors
BSDFIsotropicParams calcBSDFIsotropicParams(in ViewSurfaceInteraction interaction, in vec3 L)
{
float invlenV2 = inversesqrt(dot(V,V));
float invlenN2 = inversesqrt(dot(N,N));
float invlenL2 = inversesqrt(dot(L,L));
BSDFIsotropicParams params;
// totally useless vectors, will probably get optimized away by compiler if they don't get used
// but useful as temporaries
params.interaction.V.dir = interaction.dir.v*invlenV2;
params.interaction.N = interaction.N*invlenN2;
params.L = L*invlenL2;
// this stuff only works with normalized L,N,V
params.NdotL = dot(params.interaction.N,params.L);
params.NdotL_squared = params.NdotL*params.NdotL;
params.NdotV = dot(params.interaction.N,params.interaction.V.dir);
params.NdotV_squared = params.NdotV*params.NdotV;
params.VdotL = dot(params.interaction.V.dir,params.L);
params.LplusV_rcpLen =inversesqrt(2.0 + 2.0*params.VdotL) ;
// this stuff works unnormalized L,N,V
params.NdotH = (params.NdotL+params.NdotV)*LplusV_rcpLen;
params.VdotH = LplusV_rcpLen + LplusV_rcpLen*LdotV;
return params;
}
// get extra stuff for anisotropy, here we actually require T and B to be normalized
BSDFAnisotropicParams calcBSDFAnisotropicParams(in BSDFIsotropicParams isotropic, in vec3 T, in vec3 B)
{
BSDFAnisotropicParams params;
params.isotropic = isotropic;
// meat
params.TdotL = dot(T,isotropic.L);
params.TdotV = dot(T,isotropic.interaction.V.dir);
params.TdotH = (params.TdotV+params.TdotL)*isotropic.LplusV_rcpLen;
params.BdotL = dot(B,isotropic.L);
params.BdotV = dot(B,isotropic.interaction.V.dir);
params.BdotH = (params.BdotV+params.BdotL)*isotropic.LplusV_rcpLen;
// useless stuff we keep just to be complete
params.T = T;
params.B = B;
return params;
}Lighting (deferred, GI, forward, direct, shadowing, etc.) is the job of the renderer/programmer who knows what they are doing.
The material shaders shall provide BSDF and Importance Sampling Functions Only.
Default shader should visualize a simple non shadowed point-light of constant-world-space-intensity positioned at the eye/camera (like current example nr. 5 etc.).
If you know that your material is intended to be transparent, you can enable blending (additive or overlay/ONE_MINUS_SRC_ALPHA), I prefer precomputed RGB blend function (so you already multiply source_rgb by source_alpha in the shader).
Corollary, the fragment shader will always output a single vec4 color.
For the benefit of mirror/conductor materials, environment map lighting / IBL is allowed.
So your shader for all loaders will most probably be:
#ifndef _FRAG_MAIN_DEFINED_
#define _FRAG_MAIN_DEFINED_
void main()
{
BSDFIsotropicParams parameters;
// your BSDFSample is direction towards light (camera)
... // generate parameters from lightPos, cameraPos, viewDir and tangent-space
// might be inf by chance
vec3 totalColor = bsdf_eval(parameters);
if (specContantUsingIBL)
{
for (uint i=0u; i<specConstantMaxIBLSamples; i++)
{
vec2 _sample2d = gen_2d_sample(i);
totalColor += bsdf_cos_sample_eval(tspaceViewDir,_sample2d);
}
}
color = totalColor;
}
#endifEngine supports both models. For instance OBJ loader may flip from right hand to left hand, which means sponza looks right only with a left-hand camera attached. Depending on loader flags a flip may be performed - so the positions and normals around X axie while loading.
The example of following is
if(_params.loaderFlags & E_LOADER_PARAMETER_FLAGS::ELPF_RIGHT_HANDED_MESHES)
// perform flip on normal or position vertex