The Joomla! Extensions Directory ™


JCrawler Component

JCrawler does the same as the Google crawler. It crawls your Joomla site and stores all the urls in a XML file without any plugins. After crawling you can ping Google, Ask.com and Bing.

Features:

- sitemap generation on the fly, no additional plugins needed
- submit sitemap to 4 searchengines (google, Bing, ask.com, moreover)
- Automatic priority calculation based on internal PageRank (experimental)
- shows bad links, or not crawlable sites
- with curl: max 250 parallel connections to spider urls (configurable)
- exclude list
- modification of robots.txt with the location of the sitemap
- crawling with curl of fopen
- saves config in a xml-file

There is a new forum and a new dedicated page for JCrawler!

JCrawler is under redeveloppment to create with even more features.


Note:
- fopen or Curl is required to crawl the site

Report Extension

 

 

2009-03-08
Reviews: 1
Installed fine and seemed to work great. However after submit to Google, the crawl produced 404 error on EVERY link. I looked at the crawl report and what Google saw was different than the xml I viewed. Each link had a menu title with Upper case letters which produced a 404 error. 77 links submitted 77 404 errors. Not sure about this bug. When I view my sitemap at everything looks ok, at Google, it's different. Went back to JMSitemap.
Owner's reply

thank you for your review, you're the first with this issue. i found the bug and it will be fixed for release 1.7 greets Patrick

2009-03-04
Reviews: 4
Does exactly what it should do. Great job and thanks!
2009-02-08
Reviews: 2
Tks very much for this great component~!
2009-01-19
Reviews: 6
Does exactly what is says, easy to install easy to use, and submits to top five search engines at the click of a button. What more could you want!
2009-01-08
Reviews: 3
Awsome component. I am not a teccy and submitting site maps to the search engines I though was not going to be easy until I came across this. Fantastic - thanks. 1 button to press! how good is that?
2008-12-21
Reviews: 1
Thank you for this component. I tested many sitemap generating applications for joomla. Yours helped me to make all that work with one button pressing, without needing to search documentation or help forums. It's time saving application.
2008-12-01
Reviews: 1
thank you
JCrawler do all you want for you site map.
i had problems with others but this one satisfied me.

thanks in advance
2008-11-14
Reviews: 5
thanks for a much needed sitemap component. As intuitive as they get, yet very valuable.

Still have to see about them submissions, though ;-)
2008-11-14
Reviews: 1
I really enjoyed using this component. it develops fast and might be the next generation in sitemap creation. no plugins. no fuss...

support is great, quick, personal and friendly.

the component is getting more and more functions all the time, and i hope it will keep getting better so fast.
2008-11-11
Reviews: 1
Tried a couple of sitemap generators, a few seemed to work but google subsequently complained about formats etc. JCRAWLER WORKED FIRST TIME. Google liked it, so I like it.
Very clean page with configurable options.

Top comp and thank you very much.
2008-11-09
Reviews: 4
Absolutely the easiest sitemap component I've ever used. The automatic submission to search engines is nice too, although time will tell how well that actually works.
2008-10-14
Reviews: 1
Very good plugin, a must have in my opinion. Would be nice for future versions to give different priorities to sections or categories.
Owner's reply

yes, you're absolutly right. actually im developping an algorythm for finding the priorities.

2008-10-14
Reviews: 1
Easy install, does what it says. I was planning on spending all morning generating and submitting sitemaps to different places but with this tool I submitted to the top 5 in about 5 minutes.
2008-09-29
Reviews: 1
This works great. Easy as butter. I just ran into one glitch as one of my sites and I doubt it is related to the extension: In the Adapt to your site area my "HTTP host (readonly)" is coming up with a subdirectory that doesn't even exist, this is causing an error in the subsequent sitemap.xml file and errors within Google. Any ideas?
2008-09-24
Reviews: 1
Excellent extension! Does exactly what it's supposed to do, dead-simple install, runs quickly, even submits the sitemap for you to google, yahoo, msn, and 2 others! It actually took me longer to get my account registered here to be able to write this review than it did to install and run the extension! Highly recommend!
2008-08-31
Reviews: 1
Very nice, and helpful component. Was really easy to installand you dont even have to configure it much - just click start - generate and submit.

Looks really clean and does his Job.

Well done
2008-08-06
Reviews: 2
After trying a couple different extensions for creating Google sitemaps with little real success, I came across this little gem. It wasn't rated yet so I figured I better take care of that because it does exactly what it says. Other sitemap components generated files with errors, but not this one. You can even validate the sitemap from the admin panel. Installed in a snap and we were up and running with the Google sitemap. I did notice that there is an option to link to JCrawler from the Joomla menu items, but that seems to just throw an error, so I'm not sure what that is for, but who cares...
2008-07-14
Reviews: 20
nifty little tool. It installed with an error but the component was there, so I tried it and it generated a complete sitemap which is great. I also uploaded but I do not know if sometings is going to happen.

Thanks
Page 4 of 4