The Joomla! Extensions Directory ™


JCrawler Component

JCrawler does the same as the Google crawler. It crawls your Joomla site and stores all the urls in a XML file without any plugins. After crawling you can ping Google, Ask.com and Bing.

Features:

- sitemap generation on the fly, no additional plugins needed
- submit sitemap to 4 searchengines (google, Bing, ask.com, moreover)
- Automatic priority calculation based on internal PageRank (experimental)
- shows bad links, or not crawlable sites
- with curl: max 250 parallel connections to spider urls (configurable)
- exclude list
- modification of robots.txt with the location of the sitemap
- crawling with curl of fopen
- saves config in a xml-file

There is a new forum and a new dedicated page for JCrawler!

JCrawler is under redeveloppment to create with even more features.


Note:
- fopen or Curl is required to crawl the site

Report Extension

 

 

2013-11-18
Reviews: 11
It is very comfortable and adds all the URLs in a file sitemap.xml
You also have the option to create the robots.txt file
2013-09-05
Reviews: 1
It’s a very good and simple component, ideal for sites where contents don't change daily, automatically.
I tried Xmap before but it "looses" some pages even they are in the menu (to be xmapped, pages need a link on the menu; that's why I switched to JCrawler).
In the other hand, JCrawler has been discontinued, so if you need improvements or support, a bad thing. I understand developers, but it's a pity they decided not to continue or the fact that they don’t find collaboration.
2013-08-13
Reviews: 3
I have around 500 links on my page and it cannot crawl even with 250 parallel processes without timeout.
2013-04-20
Reviews: 1
This component has a nice layout, and appears to be a very helpful resource to make sitemap generation and submission simple. My website, which consists of over two hundred pages, generated a single url using this tool. Sitemap submission failed for both Bing and ask.com. The documentation claims to submit sitemaps to "4 searchengines (google, Bing, ask.com, moreover)", yet it only attempted to submit the sitemap to three (google, Bing, and ask.com). I have uninstalled this extension and would not recommend it to anyone.
2013-02-08
Reviews: 1
Would give five start but something didn't allow me. I have some links in the site which are not directly linked from front page but they are in the menu of back end. Even if I directly browse links they are running properly. Unline Xmap, JCrawler do not have menu selection option white making sitemap that disappointed. If there is option to select 'Menus" before making sitemap would be excellent.
Thanks!
2013-01-22
Reviews: 3
I used Xmap and it was only able to find 28 links on my site to put in the sitemap. It did it through the menu. I then tried some other fw-sitemap thing and it didn't even have an admin interface. Go figure.

JCrawler was perfect. It found 52 links, and made a nice XML file for me too! Simple and perfect. It makes me wonder how developers (others) can get simple use value so wrong.

Well done JCrawler!
2013-01-01
Reviews: 2
Very simple sitemap creator that worked prefect for my websites. Easy installation with no add-ons or fuzzy components. It does what it claims to do in a quick and simple manner.
2012-12-01
Reviews: 7
Excellent and useful component. It works without any problems or conflicts. After I discovered this component, I removed other sitemap component from all websites.
Enough said.
2012-11-20
Reviews: 8
No more fretting over how to create a site map. Easy install and easy to use. Elegant! Installed in all my sites flawlessly. Showed me all my errors to fix, too! Many thanks.
2012-10-28
Reviews: 1
This is a pretty good sitemap generator. Would like to see an option to exclude the robots.txt paths. Generated some errors from the MYSQL, but it's working anyway.

I used to work with XMAP but they messed it up completely, now it's based on the menu, instead of on the actual content on the site.

Keep the good work!
2012-10-25
Reviews: 6
I never thought I would be excited about a mapping utility!

I installed on joomla 2.5 site and in less than five minutes it had the map built and submitted.

I double checked the map to make sure it was built properly because I have had problems with other extensions that are similar and eveything checked out.

Bravo to you dear developer...this is a great piece of work.

I hope everyone donates to keep you work on this going.

My advise to anyone reading this... run this before trying anything else. It will save you a lot of time and frustration.
2012-08-29
Reviews: 10
Having experienced problems with other sitemap components, this was the perfect solution. There was a lot of my site I did not want mapped and between the robots file and the configuration of this great component it was quick and easy to configure the sitemap and submit to Bing and Google. I'm now using this on all my sites! Recommended!
2012-07-19
Reviews: 3
I have installed several extensions trying to achieve a simple sitemap to 5 sites.

After installed Jcrawler the problem was completely solved in a few seconds.

There is some minor errors. But they are releasing bug fixing versions in a few time base. So it will be 5 stars with the time.

Congratulations to the theme. And keep up the good effort.
2012-07-14
Reviews: 6
I am using Joomla 2.5, which causes problems with quite a few other sitemap tools.

This extension was simple to install and use. Functioned as advertised!
2012-06-25
Reviews: 2
Worked great and easy to use. Submission to search engines an added bonus.
2012-06-21
Reviews: 10
wonderfull, just working.
No plugins needed for virtuemart ...
2012-05-20
Reviews: 4
Your component is great, easy to use and i recommended as a "must have extension". One thing tho, modify/update the link with the new version, even though your site has the latest version. Great job, jcrawler! :)
2012-04-19
Reviews: 1
Sorry but I have removed the extension as it crashed the Apache web-service and my server became unavailable.
shame :(
2012-03-14
Reviews: 14
Never was able to get it to work, tried on my local systems and on shared hosting platforms. Tried version for Joomla 1.5 and 2.5 and nothing.
2012-02-09
Reviews: 4
I've used Xmaps in the past but with the new 2.5 installs, I have used Jcrawler. It works as advertised. I like the auto-submit to the three major search engines. Very happy!
Page 1 of 4