lesson1Title

lesson2Title

lesson3Title

lesson4Title

lesson5Title

lesson6Title

lesson7Title

lesson8Title

lesson9Title

lesson10Title

lesson11Title

lesson12Title

lesson13Title

lesson14Title

lesson15Title

lesson16Title

pythonIntroCrawlingChapter4Title

lesson17Title

pythonIntroCrawlingChapter1Title

pythonIntroCrawlingChapter2Title

pythonIntroCrawlingChapter3Title

# Key Takeaways So Far

So far, you've covered crawling project design and the first project: crawling Wikipedia.

Before moving on to crawling the Django repository, let's review the key points card by card.

The User-Agent header identifies the program (browser or crawler) sending the request, and many sites require it.

Key Takeaways So Far

Which header identifies your program when sending a crawling request?