Allocating fields on sets

[klevzoff]

Is your feature request related to a problem? Please describe. Add the capability to allocate data fields on mesh sets (e.g. element, face, vertex sets). This is needed for efficient implementation of boundary conditions that are not prescribed directly on DOFs.

Describe the solution you'd like Proposed implementation: in ObjectManagerBase -> "sets" subgroup replace stored view wrappers of index sets with another level of subgroups that will contain the index set view wrapper along with any data fields and will be properly resized. Any loop over such a set (e.g. in BoundaryConditionBase) should pass to the user callback lambda both the mesh index and the index-within-set of the cell/face/vertex, so that user code can properly access both the globally-allocated and the set-allocated fields.

Describe alternatives you've considered

• Always allocating the fields on the full mesh (wastes a lot of memory)
• Rewriting boundary condition loops so as to avoid storing data (leads to more messy and potentially slower code)

Additional context

[klevzoff]

I can try to implement this, unless there are any problems with this approach that I'm not seeing.

[rrsettgast]

@klevzoff I had been avoiding implementing subsetting because we hadn't really needed it. Your proposal is pretty much what I had in mind, but the index set is really a upward pointing map from the subset object to the parent object, and the parent object then would have a downward pointing map for the subset object. That way you would only need one of the indices (mesh index, or set index) and you could get the other. A couple of questions:

1. Do you need to synchronize any of the subset data across mpi ranks? We would need to support that for the general case, but i'm not sure about BC case.

2. I don't recall what sort of data you are required to store to implement the boundary conditions. Can you refresh my memory and describe what you need to store?

[klevzoff]

1. I don't think parallel sync is needed for BC sets. We can evaluate BCs independently on each rank directly on ghosted faces/vertices, so long as they are included into sets.

2. It's pretty much the primary solution fields (e.g. pressure) and all dependent constitutive data (e.g. density, viscosity). Whenever the BC need to be applied, the workflow is:

• call BC manager to populate primary solution field (for Dirichlet at least)
• call state update on constitutive model(s) on BC sets (we'll need to hang them there as well) to compute dependent fields
• use both in a separate assembly-like loop to apply to the matrix/rhs

[rrsettgast]

@klevzoff I'm think the thing that I keep forgetting is that for a multiphase solver, you can't just specify the boundary pressure and be done with it. You have some other solution fields that are not specified...correct?

OK. We can do subsetting. I was thinking that we derive an ObjectSubset class from ObjectManagerBase and add the upward maps to ObjectSubset, and have the ObjectSubset add downward maps to its parent object.

[klevzoff]

@rrsettgast Yes, you have to specify the component fractions as well (which, by the way, are not even out primary solution field, but rather a derived intermediate quantity). In short, it could really be any field the solver deems necessary.

I'm not really sure we need upward maps for anything, or at least for the use cases I'm thinking of. If you write a loop over subset, you always have both indices for free. If you're trying to access subset information from a global loop (over all mesh objects)... you'd have to check whether the object is in the subset first... so a lot of branching, plus the upward map (whether hash-table or tree based) will have overhead. So might be a good idea to avoid that pattern anyway.

This is fairly low priority for me, btw. It's probably going to be a while before we get to flow problems of the size where memory is a constraint, so I can keep using full-mesh fields for now.