Hi im coding to code a Tool that searchs for Dirs and files.
have done so the tool searchs for dirs, but need help to make it search for files on websites.
Any idea how it can be in python?
You cannot get a directory listing on a website.
Pedantically, HTTP has no notion of directory.
Pratically, WebDAV provides a directory listing verb, so you can use that if WebDAV is enabled.
Otherwise, the closest thing you can do is similar to what recursive wget does: get a page, parse the HTML, look for hyperlinks (a/@href in xpath), filter out hyperlinks that do not point to URL below the current page, recurse into the remaining urls.
You can do further filtering, depending on your use case, such as removing the query part of the URL (anything after the first ?).
When the server has a directory listing feature enabled, this gives you something usable. This also gives you something usable if the website has no directory listing but is organized in a sensible way.
Search Files& Dirs on Website