JCrawler
Version
1.13 Stable (last update on Jun 24, 2012)
Rating
Compatibility
Reviews
75
License
GPLv2 or later
Non-Commercial
Type
Date Added
13 July 2008
Features:
- sitemap generation on the fly, no additional plugins needed
- submit sitemap to 4 searchengines (google, Bing, ask.com, moreover)
- Automatic priority calculation based on internal PageRank (experimental)
- shows bad links, or not crawlable sites
- with curl: max 250 parallel connections to spider urls (configurable)
- exclude list
- modification of robots.txt with the location of the sitemap
- crawling with curl of fopen
- saves config in a xml-file
There is a new forum and a new dedicated page for JCrawler!
JCrawler is under redeveloppment to create with even more features.
Note:
- fopen or Curl is required to crawl the site
Worked first time, created every single link on the site whereas xmap would only create the menu links for some reason. This is a great little extension. Well done to the developer, clean, neat and tidy xml file produced. That to me is the BEST part, it produces a physical XML file and not a lengthy component link name.
I just can't be unfair while rating this tool, however i've not been successful to implement it. it seems to just sit there and keep crawling. Is it because my site is very large, it host 21368 links as of now.
May be i am having hard luck :(
May be i am having hard luck :(
Hi,
this is very good extention like very much. I tried on small page it worked excellent. But I have bigger portal there was fatal error.
I tried to crawl but memory was not enough, so how will I solve it I do not know.
I really like this extention and want to use on my portal.
Thank you.
this is very good extention like very much. I tried on small page it worked excellent. But I have bigger portal there was fatal error.
I tried to crawl but memory was not enough, so how will I solve it I do not know.
I really like this extention and want to use on my portal.
Thank you.
This was extremely easy to install and set up. Sitemap was submitted to the search engines within minutes and Google crawled all the site pages the same day.
I have yet to see any of the other search engines make an appearance... but nevertheless google works fine for me!
I have yet to see any of the other search engines make an appearance... but nevertheless google works fine for me!
This is a first rate component! Creates a beautiful xml site map. You can easily specify what you want the crawler to ignore and with 1 click of the mouse it will submit your site map to selectable list of major search engines. This will be standard in all of my builds from now on.
There was 1 tiny glitch, if you want your site map to perfectly pass validation you will want to switch the following lines of code...
[review system strips tags out from code; code removed]
NOTE: This is addressing release 1.7Beta. Depending on when you read and download this extension this may have been addressed. Check the file against the above before and after to see.
There was 1 tiny glitch, if you want your site map to perfectly pass validation you will want to switch the following lines of code...
[review system strips tags out from code; code removed]
NOTE: This is addressing release 1.7Beta. Depending on when you read and download this extension this may have been addressed. Check the file against the above before and after to see.
Owner's reply
Thanks for your cool review!
The validation issue is fixed in the latest build available on the jcrawler website.
Soon I'll release the next version!
No further words needed before the title.
Made my sitemap in few secconds, much better than the most used tools arround the web.
Congratulations.
Made my sitemap in few secconds, much better than the most used tools arround the web.
Congratulations.
Since i am not on php4 i cannot say an opinion on that, but i am on PHP5, and what is confusing is that it installs fine, and also it crawls and says everything is a sucsess.
Of course the sitemap.xml file that is generated is very empty though.
Hope it will work with PHP, because this is what i am looking for.
Morten
Of course the sitemap.xml file that is generated is very empty though.
Hope it will work with PHP, because this is what i am looking for.
Morten
Owner's reply
Hey Morten, thx for your review.
Unfortunatly has JCrawler in version 1.7 a problem with relative links. So thats why your sitemap is empty. It will be fixed in version 1.8
Greets Patrick
Nice component! Unfortunately it does not works for me with PHP5 enabled. But if I disable and switch back to PHP4.x, the sitemap generation works like a charm!
Owner's reply
Thank you! The next version will be more compatible with different PHP versions/configurations.
Installed fine and seemed to work great. However after submit to Google, the crawl produced 404 error on EVERY link. I looked at the crawl report and what Google saw was different than the xml I viewed. Each link had a menu title with Upper case letters which produced a 404 error. 77 links submitted 77 404 errors. Not sure about this bug. When I view my sitemap at everything looks ok, at Google, it's different. Went back to JMSitemap.
Owner's reply
thank you for your review, you're the first with this issue. i found the bug and it will be fixed for release 1.7 greets Patrick
Does exactly what it should do. Great job and thanks!
Page 3 of 4
