WP Content Crawler
Get content from almost any site to your WordPress blog, automatically!
For what it can be used
- Create a personal site which collects news, posts, etc. from your favorite sites to see them in one place
- Use it with WooCommerce to collect products from shopping sites
- Collect products from affiliate programs to make money
- Collect posts to create a test environment for your plugin/theme
- Collect plugins, themes, apps, images from other sites to create a collection of them
- Keep track of competitors
How it works
It’s all about CSS selectors and you can learn how to use them in minutes by watching the introduction tutorial.
See it in action, learn in minutes
Available in two languages, ready for translation
PHP >= 5.5, mbstring
Tested with WP versions:
4.5.3, 4.4.2, 4.3.3, 4.2.7, 4.1.10, 4.0.10, 3.9.11
- Save post’s title, excerpt and content.
- Save meta keywords and description.
- Save images in the posts to your site.
- Save featured image of target post as the featured image of the post created in your site.
- Save the list in posts (extract the list from the post).
- Save paginated posts. If target post is paginated, the post in your site can be paginated as well.
- Save anything from target page as post meta.
- Find and replace anything in the target page’s HTML before saving its content.
- Find and replace using regular expressions.
- Find and replace in image URLs before saving them.
- Manipulate target page’s HTML before saving to make it suitable for your needs.
- Remove unnecessary elements from target page’s HTML.
- Map categories of your site to target site’s categories to keep posts organized.
- Automatically get category URLs for category mapping.
- Add unlimited categories for a site and automatically check for new posts uniformly.
- Set alternative CSS selectors for each setting. By this way, get content from the pages of the same site which are designed differently.
- Automatically save posts.
- Automatically check for new posts.
- Set maximum number of category pages to be checked automatically.
- Add unlimited sites.
- Import/export site settings.
- Set post type. For instance, you can add posts as products if you use WooCommerce.
- Set a template for the post.
- Set a template for each list item.
- Set passwords for created posts.
- Keep notes for each site to keep a changelog or anything you want.
- Test any CSS selector and find-and-replace setting right away.
- Test a site before activating it for automatic crawling.
- Manually create posts by providing the URL of the posts.
- If you want, use settings different than general settings for each site.
- Activate/deactive each site any time to start/stop automatic crawling.
- Either directly publish the posts or keep them as drafts to make changes before publishing.
- Online documentation
- Video tutorials for a quick start
- Ready to translate (.po file)
- Get updates from your WordPress admin panel.
- Save images as gallery. If you use WooCommerce, you can save images from target page as product gallery.
- Get anything from target page as a shortcode and use the shortcodes in templates to place any data anywhere you want.
- Save tags using CSS selectors.
- Find and replace in post tags.
- Limit the number of tags that can be added to a post.