Day 12 of making an autonomous G1 bartender. First thing is first, the policy on the real G1 is now 30s in total, just like in sim. However, our sim policy is still not optimal and will not pick up all cans. Because we dont have much time and need to get this working in the next 5 days and RL will just not work if hyperparams are off, We decided to rent out a bunch of ~25 5090s (they are under a dollar each) and run a massive number of Protein (from pufferlib) sweep variations. We manually tested out simple things like changing the reward values from completing an entire task and we got completely different success rates within 24 hours. The default rewards that astra gave us were not ideal. So, with this Protein sweep, we have a single 5090 as an orchestrator and 4 other 5090s computing different starting values and this gives us 5 batches of sweeps. I have decided that treating the reward numbers could also improve the training, so by tmrw, I really hope that we get good hyperparameters so that my sanity could remain :)

Day 11 of making an autonomous G1 bartender. We started to rent some 5090 gpus because for some reason, renting 60 5090s is cheaper than an hour of using codex astra medium. However, I only rented 4 becuase making the 5090s communicate with each other is tough. Today was the first time we had the entire policy from the very start to the very end go through on the G1. Unfortunately, it took 23 MINUTES TO EXECUTE. It was supposed to take around 3 minutes and that was already too long. Let me give you guys some reasons as to why it took so long. The configured 40 Hz was only a target. Deadline enforcement had been disabled, so slow iterations continued instead of aborting; the runtime did not maintain 40 Hz. Camera segmentation commonly took 38-44 ms, already longer than the 25 ms budget for an entire 40 Hz iteration. DDS body/hand publication sometimes took 25–53 ms. That could consume or exceed the complete 25 ms period before inference and safety checks were included. Camera-frame or embedding unavailability produced hold iterations: the robot received another hold command, but the learned policy did not advance one step. Body and Dex3 state synchronization occasionally exceeded its skew threshold, causing state acquisition retries and additional waiting. DDS timing-gap rejection could retry the same command. Those retries consumed time without advancing the policy counter. Entry and ownership transfer added approximately 20–25 seconds, but this explains only a small portion of the 23 minutes. Vision encoding, GRU inference, state validation, DDS publication, and scheduling ran in the same control pipeline. Their combined latency accumulated on every one of the 7,722 steps. The observed 25–53 ms DDS and 38–44 ms segmentation times alone do not fully explain an average of 179 ms per policy step. There must also have been repeated waits, retries, holds, or another unmeasured blocking section. The old run did not have per-stage timing telemetry, so an exact millisecond breakdown cannot be reconstructed retroactively. The newly deployed runtime now logs effective Hz, hold behavior, camera fetch/decode, segmentation, vision encoding, policy computation, state guards, DDS publication, scheduler delay, and complete tick latency. The next run will show precisely where the missing approximately 154 ms per step is going. It is actual insanity how many things could go wrong. fortunately, this task doesnt really change its state too much so if things are slower than anticipated such as not recieving a frame until 250 ms later, it should still be fine.

Day 9 of making an autonomous G1 Bartender. It appears that when trying to transfer a policy from sim to real, the joints moved too quickly. I thought that this was a simple fix and to just create an adapter to move the joints below a certain threshold but there needed to be a lot more converting than anticipated. I went back to the sim -> behavior cloning -> humanoid movement copies -> astra controlling the G1 and realized that the original ~67 samples that were provided had really bad flicks and that we may need to redo the entire process. I made astra generate a new episode of picking up a can in sim and tested it out on the G1. After changing the limitations that astra set on the joint limits, I finally got the G1 to move acording to the controlled sim (not policy). Now I'll be generating ~30 episodes with astra and restart the entire training process. The majority of the morning was spent redesigning sim to real. I kept asking what the inputs to the policy were and astra constantly told me that it was RGB-D (i thought normal images + depth), joint velocities, accelerations, and their position (the robot knows this in real life too). But there was actually an entire processing step trying to get the hand to can locations and the RGB image was not being fed into the network at all. All gpts do is lie, smh.