4 Days to 15 mins
I did a very similar task few days ago wherein a record processing program was supposed to do pivoting, and when increased from 100 to 1000, 4000 records, the time went exponentially increasing up to 4 days. So, yes, loops were a problem, but not the only problem. After, a careful inspection I found that 2 things could be fixed, and that after 6 days of work, reduced the time for 1000 records to just over 15mins, and 2000 to just over 30 mins. We checked the progress every 2 days to see where we headed, and finally solved it.
What did we achieve?
1) Where the emoloyer expected and posted a project to reduce the time from 4 days to around 15 to 20 hours, I did it under 2 hours.
2) The code was significantly smaller
3) It ran in linear time, now
4) it could do so in 4gb, that failed earlier even with 32gb ram
Have you checked these?
So, we could certainly look at the while loop, but if a loop is inevitable, then there may be other things...like time complexity of the steps, using another more optimized function for sorting a vector than one used, vectorizing parts as much as possible, working with sparse matrices, too many function calls, reading writing speed, and many more...although not comprehensive, but some common mistakes.
How do we go about it?
1) Kindly, attach your (complete) code (is it everything in the attachment??) so that I can take a look at it. Attach the objective brief PDF or docx.
2) Then I'll need a session or a video call where you must explicitly explain what you're trying to achieve, not what the code is trying to do, and **
3) then let's figure out what the code is trying to do and if it can be done better?**
If that makes sense, kindly message me whenever you're free, if I'm not online, leave your queries and I shall reply ASAP.
Thank you!
** means very important