For my app, how many threads would be optimal?
It's usually simpler to make multiple concurrent processes.  Simply use subprocess to create as many Popens as you feel it necessary to run concurrently.
There's no "optimal" number.  Generally, when you run just one crawler, your PC spends a lot of time waiting.  How much?  Hard to say.
When you're running some small number of concurrent crawlers, you'll see that they take about the same amount of time as one.  Your CPU switches among the various processes, filling up the wait time on one with work on the others.
You you run some larger number, you see that the overall elapsed time is longer because there's now more to do than your CPU can manage.  So the overall process takes longer.
You can create a graph that shows how the process scales.  Based on this you can balance the number of processes and your desirable elapsed time.
Think of it this way.  
1 crawler does it's job in 1 minute.  100 pages done serially could take a 100 minutes.  100 crawlers concurrently might take on hour.  Let's say that 25 crawlers finishes the job in 50 minutes.
You don't know what's optimal until you run various combinations and compare the results.
I have a simple Python web crawler. It uses SQLite to store its output and also to keep a queue. I want to make the crawler multi-threaded so that it can crawl several pages at a time. I figured i would make a thread and just run several instances of the class at once, so they all run concurrently. But the question is, how many should i run at once? should i stick to two? can i go higher? what would be a reasonable limit for a number of threads? Keep in mind that each thread goes out to a web page, downloads the html, runs a few regex searches through it, stores the info it finds in a SQLite db, and then pops the next url off the queue.