Hey,
I came across this project and this interests me.
Talking about the Project, Since the website is to be scraped every second and to be honest, scraping a page can be slow due to various factors (internet speed, etc.), what we can do is we can actually use multiprocessing where each second, it will scrape the data as a background process. Like this, we can scrape data in real time each second without missing out any data.
As for the scraping part, that's relatively easy. I have already used the real time scraping and multi processing before. You can check my portfolio on my profile too!
As for calling the API for Markets with the scraped Data, I've achieved that in my crypto trading bot and the best possible way to do it is to push the scraped data directly into the next function. That doesn't produces and lags and the script works extremely fast.
As for saving it into an Array, that's easy too. We just have to create an array in the Global scope and append the data in it at the end of the function.
I've mapped the whole script after reading the specifications and we discuss it further if you'd like. Feel free to leave a message. Looking forward to Collaborate.
Cheers,