JUNOSW + Opticks : Status and Plan

JUNOSW + Opticks : Status and Plan

Simon C Blyth, IHEP, CAS — Simulation Meeting — 2 July 2024


Geant4 + Opticks + NVIDIA OptiX : Hybrid Workflow

https://bitbucket.org/simoncblyth/opticks

Opticks API : split according to dependency -- Optical photons are GPU "resident", only hits need to be copied to CPU memory


Geometry Model Translation : Geant4 => CSGFoundry => NVIDIA OptiX 7+

Geant4 Geometry Model (JUNO: 400k PV, deep hierarchy)

PV G4VPhysicalVolume placed, refs LV
LV G4LogicalVolume unplaced, refs SO
SO G4VSolid,G4BooleanSolid binary tree of SO "nodes"

Opticks CSGFoundry Geometry Model (index references)

struct Notes Geant4 Equivalent
CSGFoundry vectors of the below, easily serialized + uploaded + used on GPU None
qat4 4x4 transform refs CSGSolid using "spare" 4th column (becomes IAS) Transforms ref from PV
CSGSolid refs sequence of CSGPrim Grouped Vols + Remainder
CSGPrim bbox, refs sequence of CSGNode, root of CSG Tree of nodes root G4VSolid
CSGNode CSG node parameters (JUNO: ~23k CSGNode) node G4VSolid

NVIDIA OptiX 7+ Geometry Acceleration Structures (JUNO: 1 IAS + 10 GAS, 2-level hierarchy)

IAS Instance Acceleration Structures JUNO: 1 IAS created from vector of ~50k qat4 (JUNO)
GAS Geometry Acceleration Structures JUNO: 10 GAS created from 10 CSGSolid (which refs CSGPrim,CSGNode )

JUNO : Geant4 ~400k volumes "factorized" into 1 OptiX IAS referencing ~10 GAS


n-ary CSG Compound "List-Nodes" => Much Smaller CSG trees

Generalizes binary to n-ary CSG trees

  +---------------+----------------------+
  | 2-ary tree    |   n-ary "list-node"  |
  +===============+======================+
  |        U      |      U[A,B,C]        |
  |       /       |                      |
  |      U        |                      |
  |     / \       |                      |
  |    U   A      |                      |
  |   / \         |                      |
  |  C   B        |                      |
  +---------------+----------------------+
Hints in G4VSolid name : Specify shape more precisely
=> better suited intersect alg => less resources => faster
eg: CSG_DISCONTIGUOUS Union
user guarantees no overlaps, eg "union of holes" to be CSG subtracted : => simple, low resource intersect

cxr_min__eye_1,0,0__zoom_1__tmin_0.5__World_solid.jpg

3x3x3 grid of MultiUnion/list-node each with 7x7x7=343 Orb
would not be possible without list-node

JUNOSW+Opticks (2024/06) : Geometry Issues Summary

WaterPoolConstruction + HBeamConstruction : Addition of DeadWater + Tyvek

FIX (MR 458, J24.1.1) under test : simplify geometry using hierarchy approach (WaterPoolConstruction__CONFIG envvar)

FastenerAcrylicConstruction

FIX (MR 468, J24.1.1) under test : using G4MultiUnion/list-node (FastenerAcrylicConstruction__CONFIG envvar)

XJfixtureConstruction + SJReceiverFasternConstruction

Calib_GuideTube_Construction


sDeadWater

"sDeadWater/sTyvekFilm" issues at WaterPool base


WaterPoolConstruction "Sibling Soup" vs HIERARCHY

"Sibling Soup" : (DeadWater, Tyvek, Air, Steel) in vetoWater

sDeadWater,sTyvekFilm : 6 plane G4Polycone with 31 holes

  +---------------------+   +----------------+---+    1 plane
  |                     |   |                    |
  +   +-----------------+   +----------------+   +    2 planes
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  +   +                   +                  +   +
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  |   |                                      |   |
  +   +--+  +---+  +------+------------------+   +    2 planes
  |      |  |   |  |                             |
  +------+  +---+  +------+----------------------+    1 plane

ISSUES : with current "Sibling soup" geometry


WaterPoolConstruction__CONFIG : testing alternative geometry

branch: blyth-WaterPoolConstruction__CONFIG-to-fix-pillar-impingement-and-opticks-translation

https://code.ihep.ac.cn/JUNO/offline/junosw/-/merge_requests/458 MERGED: 2024-06-17 (J24.1.1)

CONFIG Notes on WaterPoolConstruction HBeamConstruction
0:ASIS sibling soup => very complicated CSG
1:ZERO_HOLES skip 60 one-by-one hole subtractions from DeadWater Tyvek
2:MULTIUNION_HOLES collect holes into G4MultiUnion(listnode), subtract once
3:HIERARCHY

radical simplification, minor geometry change :

  • skip bottom angled cross piece
  • raise base of pillars within vetoWater

Set CONFIG envvar to change geometry, default is no change "sibling soup":

unset WaterPoolConstruction__CONFIG
export WaterPoolConstruction__CONFIG=0 # ASIS : same as unset

export WaterPoolConstruction__CONFIG=3 # HIERARCHY

cxr_min__eye_0,1,0__zoom_1__tmin_0.1__GZ1.B06_07_FlangeI_Web_FlangeII:0:-1.jpg

Using G4MultiUnion/listnode for 62 subtracted holes works, BUT...


cxr_min__eye_0,1,0__zoom_1__tmin_0.1__GZ1.B06_07_FlangeI_Web_FlangeII:0:-100000.jpg

Impingment at bottom of WaterPool apparent, from angled cross-piece


cxr_min__eye_0,1,0__zoom_1__tmin_0.1__sDeadWater:0:-1.jpg

Also impingement from vertical and tilted pillars


cxr_min__eye_-1,0,0__zoom_1__tmin_0.1__GZ1.A06_07_FlangeI_Web_FlangeII:0:-100001.jpg

WaterPoolConstruction__CONFIG=HIERARCHY  (much simpler)

* skip bottom angled cross piece

* raise pillar base 126mm (keep within vetoWater)


FastenerAcrylicConstruction

Problematic deep CSG tree without list-node

+------------------------------------------+
|                                          |
|                                          |
|                           U              |
|                          / \             |
|                         /   \            |
|                        /     S           |
|                       U     / \          |
|                      / \   I   J         |
|                     U   H                |
|                    / \                   |
|                   U   G                  |
|                  / \                     |
|                 U   F                    |
|                / \                       |
|               U   E                      |
|              / \                         |
|             U   D                        |
|            / \                           |
|           U   C                          |
|          / \                             |
|         A   B                            |
|                                          |
+------------------------------------------+

U : Union
S : Subtraction
A-J : Tubs (cylinder) primitive

cxr_min__eye_0,3,-1.5__zoom_1__tmin_0.5__uni1:0:0.jpg

FastenerAcrylicConstruction : hidden within support base


cxr_min__eye_0,3,-1.5__zoom_1__tmin_0.5__uni1:0:000000.jpg

FastenerAcrylicConstruction : 8 cylinders
modelled with G4MultiUnion/list-node

20240626_171555_00000_wireframe.jpg

triangulated wireframe render


XJfixture/anchor triangulated(?)

(0):xjfc:XJfixture

(1):xjac:XJanchor

Construction #
XJfixture 56
XJanchor 56

XJfixture

XJanchor

TODO: compare validation + performance:


cxr_min__eye_1,0,0__zoom_1__tmin_0.5__solidXJfixture:0:-1.jpg

XJfixture + XJanchor : "hold" GuideTube
(trying triangulated)

SJ triangulated render

(0):sjcl:SJCLSanchor [only 2, Acrylic]

(1):sjfx:SJFixture [36, Acrylic]

(2):sjrc:SJReceiver [only 8, Acrylic]

(3):sjrf:SJReceiverFastern [only 8, PE_PA]


GuideTube : Torus Triangulated

GuideTube (39*2*2 = 156 G4Torus)
split in phi segments, radius breaks

Intersect with torus expensive on GPU

Triangulation using G4Polyhedron

G4Poly..::SetNumberOfRotationSteps

  NumberOfRotationSteps
HepPolyhedron Default 24
Top Right 48
Bottom Right 480

Adjustable: precision of intersect, number of triangles

GPUs evolved for triangles => fast even with many


cxr_min__eye_1,0,0__zoom_1__tmin_0.5__sSurftube_0V1_0:0:-1.jpg

Testing triangulated GuideTube + XJ + SJ solids
(no choice for GuideTube torii, maybe some XJ + SJ can use "list-node")

solidXJfixture:27:-1

TRIMESH=1 MOI=solidXJfixture:27:-1 EYE=0,0.1,-20 ~/o/cxr_min.sh


2024/06 JUNO+Opticks Geom Fix

WaterPool Overlaps+Opticks FIXED with simpler approach

FastenerAcrylic works with G4MultiUnion/"list-node"

Testing triangulated GuideTube + XJ + SJ solids


2024/06 JUNOSW+Opticks : Status + Plan

Option Approach to equalize CPU/GPU geometry Status
--no-guide_tube triangulated with NumberOfRotationSteps precision control GOOD
--debug-disable-fa FastenerAcrylic using G4MultiUnion/"list-node" works GOOD
--debug-disable-xj XJfixture XJanchor Trying triangulated (if too much validation deviation use "listnode") ??
--debug-disable-sj SJCLSanchor SJFixture SJReceiver SJReceiverFastern

Immediate Steps : Geometry + Performance + Validation

Optimization : measure performance as:

Production Prep : auto event splitting, depending on VRAM