Serving 3706 extensions to the community
Categories
Directory Menu
Login or Register
JCrawler 
Version
1.7 Beta (last update on Apr 21, 2009)
Rating
Compatibility
Votes
59
Favoured
82
License
GPL
Non-Commercial
Type
Views
46313
Date Added
13 July 2008
Features:
- sitemap generation on the fly, no additional plugins needed
- submit sitemap to 5 searchengines (google, yahoo, msn, ask.com, moreover)
- Automatic priority calculation based on internal PageRank (experimental)
- shows bad links, or not crawlable sites
- with curl: max 250 parallel connections to spider urls (configurable)
- exclude list
- modification of robots.txt with the location of the sitemap
- crawling with curl of fopen
- saves config in a xml-file
Note:
- fopen or Curl is required to crawl the site
Now Beta 1.7 is out!
Changes:
- Automatic priority calculation based on internal PageRank (experimental)
- Performance and compatibility improvments
- Submission to search engines via curl
- fixed empty loc-tag problem
- improved link detection
byvedrann on September 26, 2009
This works great
byDelicious Simplicity on September 17, 2009
This is a first rate component! Creates a beautiful xml site map. You can easily specify what you want the crawler to ignore and with 1 click of the mouse it will submit your site map to selectable list of major search engines. This will be standard in all of my builds from now on.
There was 1 tiny glitch, if you want your site map to perfectly pass validation you will want to switch the following lines of code...
[review system strips tags out from code; code removed]
NOTE: This is addressing release 1.7Beta. Depending on when you read and download this extension this may have been addressed. Check the file against the above before and after to see.
There was 1 tiny glitch, if you want your site map to perfectly pass validation you will want to switch the following lines of code...
[review system strips tags out from code; code removed]
NOTE: This is addressing release 1.7Beta. Depending on when you read and download this extension this may have been addressed. Check the file against the above before and after to see.
Owner's reply
Thanks for your cool review!
The validation issue is fixed in the latest build available on the jcrawler website.
Soon I'll release the next version!
bycnovak on August 28, 2009
Easy to install, easy to use. Makes a great sitemap.xml, and what a great logo.
This as great potential however if you have over 3000 + article it just die seem was made for small sites.
Owner's reply
thanks for your comment, it depends on the performance of your webserver. I crawled successfully websites with 10000+ links.
bysvincent on July 30, 2009
You install it, do a slight bit of config (had to make a directory and file writable) and then you run it. It's the way software should be built.
Thanks for writing this plugin.
Thanks for writing this plugin.
bygeorgev on July 18, 2009
simly installed. simply customized. simply executed. perfect results. what to add? only one more thank you for the sitemap submission feature.
byrodrigojmz on June 17, 2009
No further words needed before the title.
Made my sitemap in few secconds, much better than the most used tools arround the web.
Congratulations.
Made my sitemap in few secconds, much better than the most used tools arround the web.
Congratulations.
byguysmiley on June 6, 2009
I recently tried this on my site of 1500+ links and had no issues whatsoever.
Note: It did take about 5 minutes to crawl my whole site (as I held my breath). But, 5 minutes later, I was a happy camper.
Thanks for a great addition!
Note: It did take about 5 minutes to crawl my whole site (as I held my breath). But, 5 minutes later, I was a happy camper.
Thanks for a great addition!
Engines couldn't find my website, but thanks to this component this problem is now solved. Even the Beta worked fast and made the job easy. Thank you!
byVampy on May 20, 2009
This seems like a very useful extension. I have only one problem / question.
Is it normal for my sitemap.xml to have just 7-8 URLs ? I mean, I have a lot of articles on my website but only a few URLs.
Is it normal for my sitemap.xml to have just 7-8 URLs ? I mean, I have a lot of articles on my website but only a few URLs.
bymmyhre on May 14, 2009
Since i am not on php4 i cannot say an opinion on that, but i am on PHP5, and what is confusing is that it installs fine, and also it crawls and says everything is a sucsess.
Of course the sitemap.xml file that is generated is very empty though.
Hope it will work with PHP, because this is what i am looking for.
Morten
Of course the sitemap.xml file that is generated is very empty though.
Hope it will work with PHP, because this is what i am looking for.
Morten
Owner's reply
Hey Morten, thx for your review.
Unfortunatly has JCrawler in version 1.7 a problem with relative links. So thats why your sitemap is empty. It will be fixed in version 1.8
Greets Patrick
byiNFERNO on April 7, 2009
Nice component! Unfortunately it does not works for me with PHP5 enabled. But if I disable and switch back to PHP4.x, the sitemap generation works like a charm!
Owner's reply
Thank you! The next version will be more compatible with different PHP versions/configurations.
byalsage13 on March 27, 2009
This is a great component that saved me so much time! From install to submission to the search engines it took 5 minutes. I'm so glad I found this component before I created my own site map by hand. Thank you!!!
bytomrocks on March 8, 2009
Installed fine and seemed to work great. However after submit to Google, the crawl produced 404 error on EVERY link. I looked at the crawl report and what Google saw was different than the xml I viewed. Each link had a menu title with Upper case letters which produced a 404 error. 77 links submitted 77 404 errors. Not sure about this bug. When I view my sitemap at everything looks ok, at Google, it's different. Went back to JMSitemap.
Owner's reply
thank you for your review, you're the first with this issue. i found the bug and it will be fixed for release 1.7 greets Patrick
byCrucialClick on March 4, 2009
Does exactly what it should do. Great job and thanks!
bysaadi78 on January 19, 2009
Does exactly what is says, easy to install easy to use, and submits to top five search engines at the click of a button. What more could you want!
byPlatothefish on January 8, 2009
Awsome component. I am not a teccy and submitting site maps to the search engines I though was not going to be easy until I came across this. Fantastic - thanks. 1 button to press! how good is that?
byalfauser on December 21, 2008
Thank you for this component. I tested many sitemap generating applications for joomla. Yours helped me to make all that work with one button pressing, without needing to search documentation or help forums. It's time saving application.
bym107 on December 1, 2008
thank you
JCrawler do all you want for you site map.
i had problems with others but this one satisfied me.
thanks in advance
JCrawler do all you want for you site map.
i had problems with others but this one satisfied me.
thanks in advance
Page 1 of 2