How to speed up training through parallelization across GPUs or CPUs in Machine Learning

We are trying to reduce total training time for one of our models. Currently CPU and GPU training is about the same: about 1 minute per epoch. When using the GPU, about half of that is just loading data onto the GPU, and another half is backpropagation. We just tried an experiment to see how parallelizable the model is -- first only on the cpu, and then on distributed cpus with multiple processes. There's a parameter called num_threads which controls cpu level parallelization, which gives us a decent gain when going to two threads, but not much beyond that. This suggests that the model isn't very parallelizable, which also implies that distributing across multiple GPUs wouldn't be much of an improvement either. Our typical flow for a customer is now: • User clicks on an ad in Facebook, goes to our website • User browses our website for a bit • User goes directly to the app store, downloads the app • Inside the app, the user makes an account and orders Currently, FB is able to track this cross-platform movement, I believe via IDFA. Unfortunately, we're soon going to be left without many users' IDFAs with the upcoming Apple changes. We'd like to be able to continue reporting a conversion event along with the specific user who triggered it, so Facebook can continue optimizing our ad campaign for users who are similar to the ones who converted. 1. Even though it is supposed to be used only for web events, can we use the conversion API for in-app events? Even if we don't have IDFA? 2. If we use the conversion API, will Facebook optimize the ad campaign with that data point? Do we need to include more than just email address and name for this to work? 3. Why does the Conversion API say it requires us to configure the Facebook pixel in addition? Can we report conversions through the API without the FB pixel? 4. If the conversion API doesn't work for us, could we instead use the offline events API to report conversions for specific users, even if we don't have the user's IDFA? Basically, I'm looking for some live troubleshooting for ideas to speed up training at this point. We're using Python 3 + PyTorch.