0. Use python 3.7/3.8, and please leave some comments and notes
1. A small crawler program on Windows; I can download the file and click an .exe for running it. This program has an interface for some simple operations: there is a search box and search button in the page, a status bar, a conversion button, an import button, and a visualization button.
2. First, I can type a word in the search box of the program’s interface.
3. According to the word I typed in step2, the program searches for the corresponding words on [login to view URL]
4. Create a list1 to document the explanations on [login to view URL] of the word searched in step2. Please notice that only level-1 explanations are needed (as shown in the jpg "无标题", the level-1 explanations of “this” are the sentences rounded by the red circles).
In list1, it should include the word, and the explanations attached to the word.
In list1, pls keep the sequences of the explanations, the sequence of the words in each explanation.
5. A word may have several level-1 explanations. Obviously, these explanations are written by other English words. Treat these words contained in these explanations as a set.
Create a list2 to include all the words in all the explanations of a single word, add these words to list1, and then repeat step4 to get the explanations of them one by one.
6. Repeat step4 and step5 until list1 and list2 doesn’t grow anymore.
Finally there should be the effect that all the words in list2 can find their level-1 explanations in list1, and all the explanations in list1 are constituted by the words included in list2.
7. (Simultaneously with 6) Record the mapping relationship words and form a database (will be useful in step 9).
For example, suggesting that I start by searching “A”, then the program will return explanation1 and explanation 2. Explanation1 is written by B, C, D; explanation2 is written by E, F, G.
Now I get a mapping between A and BCDEFG. Record this mapping relation.
Then, according to step4~6, the program will begin to search for the explanations of B, C, D, etc. And then it will find that B has a mapping relation with (perhaps) A, E, K, W, Z…
The program will also record these mapping relations, until list1 and list2 doesn’t grow anymore. Eventually, it will form a database carrying all these mapping relations.
8. When step6 and step7 are completed, the interface of the program will show me a notice “lists are prepared”. Then I can click a button on the interface to create an excel that includes list1 and list2, and to derive a file that has the database formed in step7.
I can choose where I want to save these files.
9. The program has another button for visualization. When I click the button, it can generate a picture of the mapping relations in the most recent step7 file, or in a step7 file that I choose. The visual effect should be something like the picture attached called "无标题2”.