subroutine to find neighbours of cells
Typically every element requires information from its neighbors to perform an update of its state. All such required neighbors constitute a so called tem_stencil_module. tem_build_horizontalDependencies, or the connectivity of elements on the same refinement level in the tree, are therefore essential for the stencils in typical mesh-based numerical schemes. We now sketch the method to find the connectivity of elements in the mesh with the help of the described mesh layout. In a distributed mesh, the first distinction has to be made with respect to processor ownership.
: tem_find_depNeigh does not exist any more Is the element in question tem_find_depNeigh a local or remote element?
For all identify_local_element, their actual position in the sparse mesh has to be identified for a given treeID.
A tem_path_type comparison of two nodes to decide their relative position in the ordered list of elements has to take into account the hierarchy of the mesh.
As in the actual simulation only leaves will be present, and no overlap of refinement levels is allowed, this is a sufficient determination. However, such overlapping regions might be created to enable the interpolation between different levels with virtual elements. These special elements are distinguished anyway, so this does not pose any problem in the search for neighborhood relations.
An element can be either identified by its treeID or by a tuple with four integer entries for the spatial coordinates and the refinement level: . This coordinate fully describes the spatial shape and position of an element and is important to determine spatial relations between elements. Conversion between treeIDs and coordinates can be achieved by the routines - tem_CoordOfId Coordinate from TreeID - tem_IdOfCoord TreeID from Coordinate
The spatial indices are limited by the refinement level: . To avoid undefined coordinates, movements through the mesh by additions to indices are done in a modulo() safeguard resulting in an periodic universe cube. This enables the movement in the mesh in horizontal direction by index alteration and translation into a treeID again. The described procedure is completely reversible, enabling the construction of the treeID for any given coordinate. Thus, the conversion between this coordinate and the serialized treeID encoding is fully described by the topology of the octree and the chosen space-filling curve ordering of elements, as explained above.
With this method we can describe the neighborhood of any given element independently of the actual solver by a simple list of relative offsets. In contrast to the generic horizontal relation, the vertical relation between child and parent nodes in the tree requires an interpolation operator between different refinement levels. This interpolation usually has to take into account solver specific requirements, but is otherwise quite isolated from the numerical operation on each refinement level. The TreElM library offers the solver a level-wise view, as suggested by the properties described above. To find all required neighbors in the distributed octree, the solver merely has to provide its horizontal dependencies. These are described with the help of an element specific tem_stencil_module. A stencil is basically a set of element-offsets , describing the relative positions of all required elements for a given element.
In this routine, level descriptor is allocated,
all elements in tree are added into me%elem as fluid type,
including their property, pntTID, stencil, neighID, sourceProc
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| type(tem_levelDesc_type), | intent(out), | allocatable | :: | me(:) |
neighbor list containing all the neighbours for the cells given in treeidsubset. Result of this routine |
|
| type(tem_BC_prop_type), | intent(in) | :: | boundary |
boundaries for the elements with bnd property set |
||
| type(treelmesh_type), | intent(in) | :: | tree |
subset of tree ids for which the neighbours will be specified |
||
| type(tem_stencilHeader_type), | intent(in) | :: | stencils(:) |
the given stencil |
subroutine tem_init_elemLevels( me, boundary, tree, stencils )
! -------------------------------------------------------------------- !
!> neighbor list containing all the neighbours for the
!! cells given in treeidsubset. Result of this routine
type(tem_levelDesc_type), allocatable, intent(out) :: me(:)
!> boundaries for the elements with bnd property set
type(tem_bc_prop_type), intent(in) :: boundary
!> subset of tree ids for which the neighbours will be specified
type(treelmesh_type), intent(in) :: tree
!> the given stencil
type(tem_stencilHeader_type), intent(in) :: stencils(:)
! -------------------------------------------------------------------- !
type(tem_stencilElement_type) :: tStencil
integer :: posInTree, nElemsBnd, iQQN, iLevel, nProcs, hashpos
integer :: x(4), nStencils, iStencil, elemPos, nStencilElems
integer :: indElem, minLevel, maxLevel, QQN
integer :: initlen
integer(kind=long_k) :: treeID
integer(kind=long_k), allocatable :: neighIDs(:)
integer :: addedPos
logical :: wasAdded
integer :: posInBCID
! -------------------------------------------------------------------- !
call tem_horizontalSpacer( fUnit = dbgUnit(1), before = 1 )
write(dbgUnit(1),*) 'Inside routine: tem_init_elemLevels '
! precondition for this routine: the first stencil must be compute stencil
if ( .not. stencils(1)%useAll ) then
write(logUnit(1),*)'tem_init_elemLevels:'
write(logUnit(1),*)'Error: The first stencil does not have the'// &
& 'useAll flag set'
write(logUnit(1),*)' The first one is considered the compute ' &
& //'stencil and'
write(logUnit(1),*)' needs to be set for all elements.'
call tem_abort()
end if
nProcs = tree%global%nParts
minLevel = tree%global%minLevel
maxLevel = tree%global%maxLevel
nElemsBnd = 0
nStencils = size( stencils )
! ------------- Prepare a hash table -----------------------------------
! Find a suitable hash size.
! The hash lookup is O(1), while the identification grows with
! log(nElems) and log(nProcs), yet allocation, deallocation and
! filling will cause some overhead of O(nHashes), this is an attempt to
! find a compromise of these parameters.
nHashes = int(max(tree%nElems, nProcs, 64), kind=long_k)
! @todo Find the closest smaller prime for the hash size, to minimize the
! number of collisions?
! Limit the size of the hash:
nHashes = min(nHashes, int(io_buffer_size, kind=long_k))
allocate(hash(0:int(nHashes-1)))
allocate(hash_elemPos(0:int(nHashes-1)))
! Reset the hash
hash = -1_long_k
hash_elemPos = -1
! --------------------------------------------------------------------------
initlen = ceiling(1.1 * real(tree%nElems)/real(maxlevel-minlevel+1))
call tem_alloc_levelDesc( me, minLevel, maxLevel, initlen, nStencils )
write(logUnit(5),*) 'Searching the neighbors for each element ...'
! Loop over all given stencils
stencilLoop: do iStencil = 1, nStencils
write(logUnit(6), "(' on stencil: ', I0)") iStencil
QQN = stencils(iStencil)%QQN
allocate( neighIDs( QQN ) )
! map user stencil to treelm definition
call tem_stencil_map_toTreelmDef( me = stencils( iStencil ))
if ( stencils(iStencil)%useAll ) then
! This stencil belongs to all elements in the tree (fluid stencil)
nStencilElems = tree%nElems
else
! This stencil belongs to only a subset of the tree
! (boundary and IBM stencils)
nStencilElems = stencils( iStencil )%nElems
end if
write(logUnit(6),"(A,I0)") ' nElems: ', nStencilElems
write(logUnit(6),"(A,I0)") ' QQN: ', QQN
! Loop over all the elements connected to the current stencil
! stencils( iStencil )
elemLoop: do indElem = 1, nStencilElems
! Reset the empty stencil
! Create the temporary stencil for assigning the positions of the
! neighbor treeIDs which are added in elem%neighID
! also store the original stencil it depends on (depends = iStencil )
call init( me = tStencil, &
& QQN = QQN, &
& headerPos = iStencil )
if( stencils( iStencil )%useAll ) then
! if the stencil is defined to be connected to all elements,
! then simply loop over all the fluid elements
posInTree = indElem
else
! if the stencil is defined for a subset, loop over the
! defined subset in %elem
posInTree = stencils( iStencil )%elem%val( indElem )
end if
! assign fluid element with info from the tree list and properties
! from disk
treeID = tree%treeID( posInTree )
! Get topological info about coordinates+level
x = tem_CoordOfId( treeID )
! Find position of this element in the elemList as it might be
! there already from previous stencil iterations
hashpos = int(mod( treeID, nHashes))
if (hash(hashpos) /= treeID) then ! cache miss
call append( me = me( x(4) )%elem, &
& tID = treeID, &
& pntTID = posInTree, &
& eType = eT_fluid, &
& nNeighIDs = QQN, &
& sourceProc = tree%global%myPart+1, &
& property = tree%ElemPropertyBits(posInTree), &
& pos = elemPos )
! for differentiating between fluid and ghost nodes
me( x(4) )%elem%property%val(elemPos) &
& = ibset( me( x(4) )%elem%property%val(elemPos), prp_fluid)
!write(dbgunit(1),*) "prp elems = ", me( x(4) )%elem%property%val(elemPos)
!flush(dbgUnit(1))
!stop
hash(hashpos) = treeID
hash_elemPos( hashpos ) = elemPos
else ! cache hit
elemPos = hash_elemPos( hashpos )
end if
posInBCID = -1
! Only perform the boundary check based on the treeID boundary
! information for stencils which work on all elements (so we can
! actually read out the boundary information from the tree )
if ( iStencil == 1 ) then
! First check, if I have prp_hasBnd
! If I don't take all the elements, then I can't analyze boundaries
if( btest( me(x(4))%elem%property%val( elemPos ), prp_hasBnd )) then
! If yes, there might be no neighbors
nElemsBnd = nElemsBnd +1
posInBCID = nElemsBnd
end if
end if ! useAll elements? (boundary information)
! calculate possible neighbors or get BCID
call tem_calc_neighbors( posInBCID = posInBCID, &
& boundary_ID = boundary%boundary_ID, &
& stencil = stencils(iStencil), &
& x = x, &
& neighIDs = neighIDs )
! append neighbors into elem%neighID or require list
do iQQN = 1, QQN
call append( me = me(x(4))%elem%neighID%val( elemPos ), &
& val = neighIDs(iQQN), &
& pos = addedPos, &
& wasAdded = wasAdded )
tStencil%tIDpos(iQQN) = addedPos
if ( stencils(iStencil)%requireNeighNeigh .and. neighIDs(iQQN)>0 ) then
call append( me = me(x(4))%require, &
& val = neighIDs(iQQN), &
& pos = addedPos, &
& wasAdded = wasAdded )
end if
end do
! Append the temporary stencil to the element
call append( me = me(x(4))%elem%stencil%val( elemPos ), &
& val = tStencil )
end do elemLoop ! indElem = 1, nStencilElems
deallocate( neighIDs )
end do stencilLoop ! iStencil = 1, nStencils
write(logUnit(5),*) 'Finished searching the neighbors ID for each fluid.'
if (tem_logging_isActive(main_debug%logger, 7)) then
do iLevel = minLevel, maxLevel
write(dbgUnit(1),"(A,I0)") 'Dump levelDesc%elem on level: ', iLevel
call tem_elemList_dump( me = me( iLevel )%elem, &
& compact = .true., &
& nUnit = dbgUnit(5), &
& stencil = .true., &
& string = 'after initializing level elements' &
& //' i.e. only fluids')
if( me( iLevel )%require%nVals > 0 ) then
write(dbgUnit(1),"(A,I0)") 'Dump levelDesc%require on level: ', iLevel
call tem_require_dump( me = me( iLevel )%require, &
& nUnit = dbgUnit(5), &
& string = 'after initializing level elements' )
end if
end do
end if
write(dbgUnit(1),*) 'Leave routine: tem_init_elemLevels'
call tem_horizontalSpacer( fUnit = dbgUnit(1), after = 1 )
end subroutine tem_init_elemLevels