Pilot 3D Rendering Migration Plan#

Introduction#

The solvcon pilot GUI is built on Qt 3D (the Qt6::3DCore, Qt6::3DRender, Qt6::3DInput, and Qt6::3DExtras modules). But Qt 3D has been officially deprecated, put into maintenance-only status, and dropped from the official Qt binary releases at Qt 6.8 (October 2024). The latest Qt (6.11) does not ship it, so the pilot GUI no longer builds against a stock Qt.

To move forward, we migrate to Qt Quick 3D, Qt’s supported successor. It renders through QRhi, the portable graphics abstraction shared by all of Qt 6 (OpenGL, Vulkan, Metal, D3D11). That same QRhi layer is also the low-level extension point for advanced rendering. The Qt Widgets application shell is kept, and the work is confined to the R* classes under cpp/solvcon/pilot/:

The rest of the pilot uses plain Qt widgets that do not touch Qt 3D: docking, the Python console (RPythonConsoleDockWidget), the 2D viewer (R2DWidget, RWorldRenderer2d), the manager, and the pybind11 bindings (wrap_pilot.cpp).

System design#

Qt Quick 3D maps onto the viewer’s needs as follows:

  • Custom geometry – subclass QQuick3DGeometry in C++ and feed it the same vertex / colour / index buffers we already assemble in RStaticMesh. This is a first-class, documented C++ path.

  • Per-vertex colour – supplied as a custom vertex attribute and consumed by a CustomMaterial shader, replacing QPerVertexColorMaterial.

  • Widget embedding – host a View3D in a QQuickWidget, which renders into the widget’s framebuffer and composes correctly with neighbouring widgets (unlike createWindowContainer, used today, which can draw over sibling widgets).

  • Axis-gizmo overlay – a second View3D with its own camera, instead of a hand-rolled frame graph.

  • Camera control – Qt Quick 3D ships OrbitCameraController and WasdController; our bespoke controllers can be reimplemented on top of these or with widget-level input handling.

The principal caveat is that Qt Quick 3D is QML-first. Its public C++ API (notably QQuick3DGeometry) has grown, but scene assembly is still expected to happen in QML, and a fully programmatic C++-only scene is not the intended workflow. The realistic shape of the migration is therefore a thin QML scene defining the View3D, cameras, lights, and materials, with the heavy data – mesh geometry and colour fields – pushed from C++ through types registered with QML_ELEMENT and QQuick3DGeometry subclasses. This is a structural change from today’s all-C++ scene graph, but it keeps the numerics and buffer management in C++ where they belong.

Qt Quick 3D is not a closed high-level engine: because it is itself a QRhi renderer, the same QRhi layer is the viewer’s low-level extension point for anything the scene graph and materials cannot express – custom shaders, compute, or direct draw calls – without leaving the engine or standing up a second renderer. The custom code runs on the one QRhi the whole window already uses, so it shares one graphics device, one backend, and one shader toolchain (QShader / .qsb, baked by the qsb tool already in the dependency prefix). Most custom shading – including the boundary highlight’s per-vertex colour – is met by a CustomMaterial or a post-processing Effect without touching raw QRhi at all. When that is not enough, three documented seams inject custom QRhi passes, from most to least integrated:

  1. Inside the 3D scene – QQuick3DRenderExtension plus QSSGRenderExtension (Qt 6.7+). Subclass the extension, return a QSSGRenderExtension from updateSpatialNode(), and implement prepareData() / prepareRender() / render(); a QSSGFrameData hands over the live QRhi, command buffer, render target, depth, and camera. The Standalone mode renders into a texture that can feed a material or effect; the Main mode injects into the scene’s pass (PreColor / PostColor), depth-tested against scene geometry. This is the right seam for custom shaders or compute drawn within the mesh view.

  2. Beside the View3D – QSGRenderNode (QRhi-based since Qt 6.6), a custom scene-graph node issuing QRhi draw calls inline with the 2D scene graph and composited with the View3D.

  3. Window underlay/overlay – the QQuickWindow beforeRendering / beforeRenderPassRecording (underlay) and afterRenderPassRecording / afterRendering (overlay) signals, for raw QRhi before or after the whole scene. Simplest, but with no depth mixing against the scene.

The shared QRhi is reached through QQuickWindow::rendererInterface() -> getResource(window, QSGRendererInterface::RhiResource) (see QSGRendererInterface). The trade-off is API stability: QRhi has been semi-public since Qt 6.6 – documented and usable, but with only a limited source/binary-compatibility guarantee, like the QPA classes. The render-extension types (QSSGRenderExtension, QSSGFrameData) are more volatile still: they live in the QtQuick3DRuntimeRender module under a version-pinned include path and can shift between minor releases. QSGRenderNode (seam 2) and the window signals (seam 3) are the steadier options. The Qt 6.11 build in the dependency environment already ships all three seams, so they are available without further dependency work.

PySide6 stays compatible. It ships a QtQuick3D binding that even supports subclassing QQuick3DGeometry in Python, and since R3DWidget stays a QWidget bridged through libpyside6, the migration needs no Python-side change beyond rebuilding PySide6 against the new module.

Migration plan#

The migration is staged so the pilot stays buildable and runnable at every step. The original Qt 3D viewer was never production-proven, so the move to Qt Quick 3D is also an opportunity to improve the visualization system. Each step below should be a self-contained, reviewable change.

We expect unexpected needs to surface during the migration, and will adjust the plan and this document as that happens.

1. Spike the data path#

Build a throwaway QQuickWidget hosting a View3D with a single QQuick3DGeometry subclass fed from the existing SimpleArray buffers, plus a CustomMaterial that reads a per-vertex colour attribute. Confirm an offscreen grab (QQuickWidget::grabFramebuffer) matches the Qt 3D output, and settle the C++/QML boundary – geometry providers registered with QML_ELEMENT and driven from C++, the QML scene kept thin. This de-risks the QML-first constraint before any production code changes.

2. Scaffold the Qt Quick 3D backend#

Add find_package(Qt6 ... Quick Quick3D) and a qt_add_qml_module block to cpp/binary/pilot/CMakeLists.txt, and gate the new backend behind a build flag (e.g. PILOT_QUICK3D) so the Qt 3D pilot keeps building during the port. The scdv dependency build already ships qtquick3d (contrib/dependency/*/build-scdv-*.sh). Stand up an empty View3D inside R3DWidget in place of the Qt3DExtras::Qt3DWindow container.

3. Port the mesh and boundary geometry#

Replace RStaticMesh’s Qt3DCore::QGeometry / QAttribute / QBuffer wireframe with a QQuick3DGeometry subclass (position attribute, Lines primitive), reusing the SimpleArray -> QByteArray assembly. Port RBoundary’s coloured ribbon to a ColorSemantic attribute consumed by the CustomMaterial. Validate the result against the sample meshes exercised in solvcon/pilot/_mesh.py.

4. Rebuild the scene and the axis overlay#

Recreate the scene root as a View3D with camera and lights, and port the colour-field path (R3DWidget::updateColorField, per-cell triangles) and the world geometry (RLines / RVertices, refreshed through updateWorld). Replace the hand-rolled frame-graph axis gizmo with a second View3D overlay on its own orthographic camera, porting RAxisMark’s QConeMesh arrows and QExtrudedTextMesh labels to Qt Quick 3D models and text.

5. Restore camera interaction#

Reimplement ROrbitCameraController and RFirstPersonCameraController on the Qt Quick 3D OrbitCameraController / WasdController or on widget-level input, preserving the current key/mouse bindings and the fitCameraToScene framing.

6. Reconnect the Python bindings and tests#

Keep the pybind11 R3DWidget API (updateMesh, showMesh, showBoundary, updateColorField, showMark, updateWorld, and the pixmap grab) unchanged in wrap_pilot.cpp, repointing the grab at QQuickWidget::grabFramebuffer. Confirm make run_pilot_pytest and the GUI tests (tests/test_pilot_mesh_info.py) pass on the new backend.

7. Cut over and remove Qt 3D#

Make Qt Quick 3D the default, delete the Qt 3D classes and the Qt6::3D* find_package calls from cpp/binary/pilot/CMakeLists.txt, drop the build flag, and remove the Qt 3D modules from the dependency build. The pilot stays runnable throughout, so this final step only retires code that nothing uses any more.

References#