Jayesh K. Gupta

Try fixing sysid example

6

I can' figure out where `ω_finite_difference` is defined (or what's the equivalent function now)? Current error: ```julia ERROR: LoadError: UndefVarError: ω_finite_difference not defined Stacktrace: [1] toss2z(toss::Vector{Vector{Float64}}, timestep::Float64; s::Int64) @ Main...

[WIP] ReinforcementLearning.jl integration

2

I realized that `CommonRLInterface.jl` never settled on what to do with continuous action spaces, so directly integrating with RLBase from ReinforcementLearning.jl. Will add tests and examples with [PPO](https://juliareinforcementlearning.org/docs/rlzoo/#ReinforcementLearningZoo.PPOPolicy) and [DDPG](https://juliareinforcementlearning.org/docs/rlzoo/#ReinforcementLearningZoo.DDPGPolicy-Tuple{}).

https://github.com/dojo-sim/Dojo.jl/blob/c513e78659de07fb930148b63989a8d5bfa72dd4/examples/reinforcement_learning/algorithms/ars.jl#L119-L127 The distributed version updates a different `normalizer` for each run. Not sure if that was intentional.

e.g. ```julia size(reset(antenv), 1) == antenv.observation_space.n ``` returns `false`. I'll add those tests in a PR. Reason is because although wedefine observation space as https://github.com/dojo-sim/Dojo.jl/blob/main/environments/ant/methods/env.jl#L44-L51 but then `get_observation` concatenates the...

See `PyTorch` for example https://pytorch.org/docs/stable/generated/torch.searchsorted.html. While the following "works", it's not really performant ```julia using MLUtils: unbatch, stack function batched_searchsortedlast(xp, x) xpb = unbatch(xp) xb = unbatch(x) res = map(xpb,...

Reason for using `Zygote.@adjoint` instead of ChainRules `rrule`

3

Would it be possible to use ChainRules `rrule` to allow supporting alternative AD engines?

Related projects of interest

1

In the JuliaPOMDP ecosystem we find it quite useful for checking the required interface for particular solvers and problems: https://github.com/JuliaPOMDP/POMDPTesting.jl

### 🐛 Describe the bug MPRS version: ```python train_dp = IterableWrapper(train_ds).batch(batch_size=batch_size).collate(collate_fn=encode_processor) rs = MultiProcessingReadingService(num_workers=num_workers) train_dl = DataLoader2(train_dp, reading_service=rs) for batch_idx, batch in enumerate(train_dl): print(batch_idx) if batch_idx > 500: break ```...

initialization not needed anymore

8

Was getting this error: ``` AttributeError: 'PDFDocument' object has no attribute 'initialize' ```