How to Import External Data Using Drupal Feeds: A Step-By-Step Tutorial

The Drupal Feeds module is a powerful tool that simplifies the process of importing and managing external content into your Drupal website. Whether you need to migrate content, update product catalogs, or aggregate news feeds, this module handles it all efficiently.

Its user-friendly interface makes data mapping and scheduling imports accessible even for non-technical users. In this blog, we will cover the key components of Drupal Feeds, walk you through the configuration steps to import data. Additionally we’ll dive into the best practices that Drupal development experts follow when working with feeds. With that said, let’s start!

What is the Drupal Feeds Module and Why Use it?

Drupal Feeds module is a widely used tool for importing, aggregating, and managing content within a Drupal site. It allows users to pull data from external sources, such as RSS feeds, CSV files, XML data, and other structured formats, directly into Drupal content types, users, or taxonomy terms.

This eliminates the need for complex coding, making it accessible to users of all technical levels. It is particularly useful for website administrators who need to automate content imports or handle large data volumes efficiently.

Why Use the Drupal Feeds Module?

Here are some of the key reasons why you might want to use the Drupal Feeds module:

• Ease of Use: The module provides a straightforward and intuitive interface for configuring and managing data imports, making it easy for users without extensive technical knowledge.
• Flexibility: It supports a wide range of data formats, allowing you to import data from various sources, whether it’s a local file, a URL, or a directory.
• Automation and Scheduling: You can schedule imports to run automatically via cron jobs, ensuring that your content stays updated without manual intervention.
• Customization and Extensions: The module can be extended with other contrib modules like the Feeds Tamper module. That allows you to manipulate and transform the data before it is imported.
• Data Integrity and Management: The module offers options to handle existing content items, such as updating, replacing, or keeping them as is. It also allows you to specify what to do with items that are no longer present in the feed, ensuring data integrity and consistency.

In short, by using the Feeds module, you can streamline your content management process and enhance the overall functionality of your Drupal website.

What are the Main Components of Feeds Module?

The Drupal Feeds Module has a well-defined architecture with three main components: Fetcher, Parser, and Processor. Each component plays a critical role in the data import process, enabling users to fetch, parse, and process external data seamlessly into Drupal’s structure. Here is the explanation of each component:

Fetcher

Role: Fetchers retrieve data from the specified source, whether it’s a URL, file, or other location.

Types of Fetcher:

• HTTP Fetcher: Fetches data from a specified URL, useful for sources like RSS feeds or API endpoints.
• File Fetcher: Used to upload files manually or retrieve files from a server directory (e.g., CSV files).

Configuration: Fetchers can be configured based on how frequently data is imported and the specific location of the data source. This setup allows administrators to control where the data is coming from and how often it’s updated​.

Parser

Role: After fetching, parsers read and interpret the format of the fetched data. That allows the module to understand and extract specific data fields.

Supported Formats:

• CSV Parser: Useful for structured data in comma-separated values.
• RSS/Atom Parser: Works with RSS feeds for content aggregation.
• XML Parser: Handles XML files for structured data, which is often used for more complex data sets.

Configuration: Parsers need to be set according to the format of the incoming data. For example, in CSV imports, users must specify delimiters and manage header rows. With XML or RSS, the parser identifies data tags and structure.

Processor

Role: The Processor handles the creation, updating, and management of Drupal entities (such as nodes, users, or taxonomy terms) based on the imported data.

Entity Types:

• Node Processor: Maps incoming data to Drupal content types.
• User Processor: Creates or updates user profiles from data sources.
• Term Processor: Imports taxonomy terms, helping structure data within Drupal.

Configuration: The Processor allows for setting rules on how the data is managed after being parsed. For instance, it can specify unique identifiers to avoid duplicate entries, define update rules, and determine what to do with obsolete data. This control helps maintain data integrity and relevance over time​.

How do These Components Work Together?

In the Feeds Module workflow:

• Fetcher retrieves data from the designated source.
• Parser then interprets the data format.
• Processor creates, updates, or manages the entities in Drupal based on the parsed data.

By configuring these components, users can create robust and automated data import workflows, tailored to support a wide range of data types and formats within a Drupal site.

By understanding these components and their interactions, you can effectively configure the Feeds Module. Using the Feeds module you can create automated data import workflows, customizable for different import needs, from simple CSV imports to RSS aggregation tasks. Now that we understand the components and working of the Drupal Feeds module, let’s learn how to configure it.

How to Configure Drupal Feeds Module and Import Data?

Installing and configuring the Drupal Feeds module involves several steps, including downloading the module, enabling it, and setting it up. Here is a stepwise process to help you get started:

Step 1: Download and Install the Feeds Module

Visit the Drupal Feeds module project page on Drupal.org and download the latest stable release of the module. You can also do it with Composer or using drush command:

composer require drupal/feeds

Or

drush dl feeds

Once you have the file, extract the downloaded module archive and upload the extracted folder to the modules/contrib directory of your Drupal installation. After downloading, enable the module through the Drupal admin UI or use Drush:

drush en feeds -y

Step 2: Enable Required Submodules

The Feeds module includes a submodule called Feeds Admin UI, which provides the graphical interface for creating and managing importers. Enable it to access Feeds settings in the Drupal admin:

drush en feeds_ui -y

Step 3: Install Required Dependencies

The Feeds Module may have dependencies on other modules. Ensure that all required modules are installed and enabled. Common dependencies include the Ctools and Job Scheduler modules. Depending on the data format you plan to import, you may also need additional parsers (e.g., Feeds Tamper for data manipulation).

Step 4: Create a New Feed Importer

Once Feeds and its dependencies are installed and enabled, you can start creating importers:

• Go to Configuration > Web Services > Feeds Importers in the Drupal admin panel.
• Click Add Feed Importer to set up a new importer, giving it a descriptive name (e.g., “CSV Article Importer”).

Step 5: Configure Importer Components

To configure importer components you must define its settings. Here is a setting you can make for each of the component:

• General Settings: Give your importer a name and description.
• Fetchers: Choose how you want to retrieve the data. For example:
  • HTTP Fetcher: For importing data from URLs (like RSS feeds).
  • File Upload Fetcher: For importing files directly from your computer (such as CSV files).
• Parsers: Select the parser compatible with your data format:
  • CSV Parser: For CSV data with options to specify delimiters and headers.
  • RSS/Atom Parser: For RSS feed imports.
  • XML Parser: For working with XML data.
• Processors: Choose the content type or entity to which you want to map the data. Common choices include:
  • Node Processor: For importing content into Drupal nodes.
  • User Processor: For importing user accounts.
  • Taxonomy Term Processor: For importing taxonomy terms.

Step 6: Map Data Fields

Field mapping is where you connect fields from the imported data to Drupal fields in your content types. After selecting the Processor, Feeds will provide a mapping interface. Here, you can specify which columns or fields in the import file correspond to fields in Drupal. To configure field mappings:

• Go to Mapping in the Feed Importer settings.
• Map each source field (like CSV headers) to Drupal fields in your target content type, such as mapping “Title” to the node title field.

Step 7: Handling Duplicates

Under the Processor settings, you can configure how Feeds should handle duplicate content:

• Skip existing items: Ignore duplicate entries.
• Update existing items: Updates entries if duplicates are found.
• Create new items: Always creates new items, even if duplicates exist.

Step 8: Running and Testing the Import

Once configured, test the import by uploading a sample file or adding a source URL, depending on your Fetcher configuration.

• Go to Content > Feeds and click Add feed for the new importer.
• Check the logs under Reports > Recent log messages if there are errors. Common issues may include data format mismatches or missing dependencies.

This setup provides a robust framework for importing various data types into your Drupal site with the Feeds module. If you are looking for a robust website that handles data effectively consider getting in touch with our Drupal development services.

Best Practices for Using the Feeds Module

When working with the Drupal Feeds module to import external data, following best practices ensures smoother imports. Here are some recommended practices for using the Drupal Feeds module effectively:

Use Feeds Tamper

Feeds Tamper enables you to preprocess or transform incoming data to fit your Drupal site’s data structure. This is useful for removing unnecessary whitespace, changing date formats, or replacing certain text values.

Tip: Keep transformations simple and consistent. For example, if you need to clean up multiple fields, create a clear order of transformations to avoid conflicts or unintended changes.

Limit the Data Imported

Importing unnecessary data can bloat your database and slow down the import process. Therefore it’s recommended to limit the data imported.

Tip: Customize your feed source or parser to retrieve only essential fields. For instance, if you only need titles, descriptions, and links, avoid pulling in extra fields like metadata or images unless required.

Configure Cron Jobs

Setting up cron jobs automates the import process, ensuring that your data stays current without manual imports. It is particularly useful for news aggregators or other frequently updated content.

Tip: Set Cron frequency based on the update rate of your data source. For example, daily imports might be enough for static data, while more dynamic data might require hourly imports.

Test Imports

Importing large volumes of data can create database clutter if errors occur. Testing in a staging environment helps you identify issues with field mapping, data formats, or transformations before importing into the live site.

Tip: Use sample datasets that mirror your full data source and test the entire Feeds importer, including Feeds Tamper settings.

By following these best practices, you can make your use of the Drupal Feeds Module more effective, manageable, and reliable. Proper planning and documentation are essential for maintaining a scalable and sustainable import system in Drupal.

FAQs About Using Drupal Feeds to Import Data

Wrapping Up

By learning how to use the Drupal Feeds Module, you can simplify your content management process. Using the Feeds module you can automate the import of data from various sources, save time, and keep your website up-to-date.

Further, you can use Drupal Feeds Tamper to modify the imported data. It is best best practice to limit the data imported and use corn jobs for streamlining the data management process.

If you are looking to build a website which is well-designed, scalable, and robust, hire Drupal developers.