
Despite there are some guides about migrating from OG to Group, as the well written https://thinktandem.io/blog/2018/03/30/migrating-drupal-7-organic-groups-to-drupal-8-group/ , and other guidelines such as https://www.drupal.org/project/group/issues/2826544 companion of https://gitlab.com/geeks4change/hubs4change/h4c/-/tree/608842e73cc02c45deede42556693a9db2a7fc37/modules_dev/h4c_migrate/config/install or https://www.drupal.org/node/2797845 . It's really adapted to your environment how you gonna carry on.
This is a short and crash guide that we at communia have done to complete the migrations. When Carrying on this migration must be estimated if the effort of writing code, in a form of migrate plugin classes or yml's, is worth enough than cooking csv's using spreadsheet skills. We've done both.
After importing everything from a standard drupal migration from 7 to 8(with what core offers), the groups types must be created, also their group roles and their group content types in a manual way. The site to be recreated is a news portal that takes headlines from several sites, and each blog or external blog is administrated in group, so being a group can be useful to add dynamic members or authors. So in the example I need to create two group types ( for each node enabled as organic group in d7), bundles are (machine)named:
blog
(it was wrongly named previously bloc). I want also to add a role for each group type as admin to add granularity in permissions. So is needed to create the role admin for each type named:blog_group-admin
forblog
group types.
I also marked the automatic role assignment for creators as admin role. It carries on a little of complexity in membership migrations as we must take care that memberships aren't duplicated (hence the plugin created below), as one membership will be created when groups are created on groups migration step, and another one when we import the memberships on memberships migration step.
So we need a simple module to add the migration configurations in yml, and it will aid us to create some extra plugins and hang on some hooks as hook_migrate_prepare_row
. The module tree:
.├── group-content-export.csv.template├── group-memberships.csv.template├── migrations│ ├── d7_node_complete_og_to_canal_blog_extern.yml│ ├── d7_node_complete_og_to_canal_proposta_externa.yml│ ├── rename_d7_node_complete_blog_post.yml│ ├── rename_d7_node_complete_podcast.yml│ ├── upgrade_d7_node_og_blog_content.yml│ ├── upgrade_d7_node_og_blog_translation.yml│ ├── upgrade_d7_node_og_blog.yml│ └── upgrade_d7_user_og_blog_memberships.yml├── og_migrate_group.info.yml├── og_migrate_group.module├── README.md└── src└── Plugin└── migrate├── process ├── GroupCreator.php ├── SkipCreator.php └── SkipOnNotEmpty.php
Go to gitlab.com to get the actual code for each file: https://gitlab.com/communia/og_migrate_group
So now let's yamling.
Groups
As Organic groups usually was assigned to nodes (if that's not the case cook it as you need), you 'll see it will be based in d7_node
migration, so d7_node
must be migrated before.
migrations/upgrade_d7_node_og_blog.yml
:
langcode: enstatus: truedependencies: { }id: upgrade_d7_node_og_blogclass: Drupal\migrate\Plugin\Migrationfield_plugin_method: nullcck_plugin_method: nullmigration_tags: - 'Drupal 7'migration_group: insert_grouplabel: 'Insert OG Bloc group'source: plugin: d7_node # The node bundle that must be get from source node_type: blocprocess: type: plugin: default_value # put blog as group type (it can be simple mapped here but this is to show how default values work) default_value: blog id: nid # The next uid will be the creator, if default role is marked it will be assigned to this user in group realm uid: node_uid label: title path: alias field_description: body # An illustration of how an image in a group could be used, images must be migrated before as usual (in this example it is not using media module). field_imatge_blogextern: plugin: sub_process source: field_imatge_blogextern process: target_id: fid alt: alt title: title width: width height: height # Here an illustration of how a term could be assigned, once again terms must exist before migrating this. field_tipus_bloc: plugin: sub_process source: taxonomy_vocabulary_7 process: target_id: plugin: entity_lookup source: tid value_key: tid bundle_key: vid bundle: vocabulary_7 entity_type: taxonomy_term revision_uid: revision_uid revision_log: log revision_timestamp: timestampdestination: # It's a group plugin: 'entity:group'migration_dependencies: null
Once module is installed this migration can be imported with:
vendor/drush/drush/drush mim upgrade_d7_node_og_blog
and rolled back with:
vendor/drush/drush/drush migrate:rollback upgrade_d7_node_og_blog
if error locks any migration state saying it's currently importing (when is not) it can be fixed with:
drush migrate:reset-status upgrade_d7_node_og_blog
It's common that sometimes a crufty translation exists, in this example we found two extra fields that covered the translation, instead of using entity_translate or node_translate so field_titolbloces
means title
field in spanish and field_descripciobloces
means field_description
in spanish . This configuration will update existing groups translating it. Must be noted that group type must be marked as translatable editing the group type settings. migrations/upgrade_d7_node_og_blog_translation.yml
:
langcode: enstatus: truedependencies: { }id: upgrade_d7_node_og_blog_translationclass: Drupal\migrate\Plugin\Migrationmigration_tags: - 'Drupal 7'migration_group: insert_grouplabel: 'Insert OG blog group translation'migration_dependencies: required: - upgrade_d7_node_og_blogsource: plugin: d7_node node_type: 'bloc'process: # Will update the previously imported groups id: plugin: migration_lookup source: nid migration: upgrade_d7_node_og_blog # Where can be found the label in spanish label: plugin: get source: field_titolbloces # Where can be found the description in spanish field_description: plugin: get source: field_descripciobloces # default value of langcode langcode: plugin: default_value default_value: es type: plugin: default_value default_value: blogdestination: plugin: 'entity:group' translations: true
This migration can be imported with:
vendor/drush/drush/drush mim upgrade_d7_node_og_blog_translation
and rolled back with:
vendor/drush/drush/drush migrate:rollback upgrade_d7_node_og_blog_translation
if error locks any migration state saying it's currently importing (when is not) it can be fixed with:
drush migrate:reset-status upgrade_d7_node_og_blog_translation
Memberships
As Organic groups memberships usually was assigned to users (if that's not the case cook it as you need), you 'll see it will be based in d7_user
migration, so d7_user
must be migrated before.
I found that here is easier to use csv, than to carry on too much code to take from here and there, so I have created a view of og memberships with data export in source site. View fields are:
- OG membership: Og membership ID (id)
- OG membership: Entity id (uid)
- OG membership: Group ID (gid)
- (Content entity referenced from og_user_node) Content: Identificator (UID) of the author (gid_uid)
View relationships are:
- OG user node target id. (Content entity referenced from og_user_node) : A bridge to the Content entity that is referenced via og_user_node
- OG membership: User from OG membership (REQUIRED)
If you prefer the sql of that view was:
SELECT og_membership.id AS id, og_membership.etid AS og_membership_etid, og_membership.gid AS og_membership_gid, node_og_membership.uid AS node_og_membership_uid, 'membership_export:page' AS view_nameFROM{og_membership} og_membershipLEFT JOIN {node} node_og_membership ON og_membership.gid = node_og_membership.nidLEFT JOIN {field_data_og_user_node} node_og_membership__field_data_og_user_node ON node_og_membership.nid = node_og_membership__field_data_og_user_node.og_user_node_target_id AND (node_og_membership__field_data_og_user_node.entity_type = 'user' AND node_og_membership__field_data_og_user_node.deleted = '0')LEFT JOIN {users} og_user_node_node ON node_og_membership__field_data_og_user_node.entity_id = og_user_node_node.uidLEFT JOIN {field_data_og_group_ref} node_og_membership__field_data_og_group_ref ON node_og_membership.nid = node_og_membership__field_data_og_group_ref.og_group_ref_target_id AND (node_og_membership__field_data_og_group_ref.entity_type = 'user' AND node_og_membership__field_data_og_group_ref.deleted = '0')LEFT JOIN {users} og_group_ref_node ON node_og_membership__field_data_og_group_ref.entity_id = og_group_ref_node.uidINNER JOIN {users} users_og_membership ON og_membership.etid = users_og_membership.uid AND (og_membership.entity_type = 'user')LEFT JOIN {profile} profile_users ON users_og_membership.uid = profile_users.uid
Anyway it must output what's in group-memberships.csv.template. Here the csv fields
id
The id of the membership(rarely used, but preserved)uid
, The user id of the membership (so the member).gid
, The id of the group that user is member.gid_uid
, The creator of the group (not used in migration as we take it with a dedicated migration plugin).
"id","uid","gid","gid_uid""46","1","14829","1""50","4","14814","4""52","12","14834","1"
And here the migration that carries on:
langcode: enstatus: truedependencies: { }id: upgrade_d7_user_og_blog_membershipsclass: Drupal\migrate\Plugin\Migrationfield_plugin_method: nullcck_plugin_method: nullmigration_tags: - 'Drupal 7 og'migration_group: insert_grouplabel: 'Insert Group Membership'source: plugin: csv # Must be obtained from source with some exporter (e.g. views data export) and placed in this path path: 'public://imports/group-export.csv' header_row_count: 1 ids: - id column_names: - id: Identifier - uid: UserID - gid: Group delimiter: ',' enclosure: '"'process: label: uid group_creator: # TO illustrate a plugin (not useful in the process), that transforms from a group id to the uid of the group author. - plugin: group_creator source: gid# -# plugin: callback# callable: var_dump # Now will check that group is there via previous migration lookup (the first one we did): gid: - plugin: migration_lookup migration: upgrade_d7_node_og_blog no_stub: true source: gid - plugin: skip_on_empty method: row message: 'gid is missing' entity_id: # To illustrate how the previous `group_creator` could be injected here( not useful): #- # plugin: callback # callable: var_dump # source: "@group_creator" # The plugin created to stop creating membership if current row user uid is the same as the group creator, to prevent duplicated members: - plugin: skip_creator user_key: uid gid_key: gid method: row # Finally check if user uid exists (probably if we are in this process point): - plugin: migration_lookup migration: upgrade_d7_user no_stub: true source: uid - plugin: skip_on_empty method: row message: 'uid is missing' # We hard code the role for all the migrated members: group_roles: plugin: default_value default_value: blog-group_admin type: plugin: default_value default_value: blog-group_membershipdestination: # Memberships are considerated group_content also. plugin: 'entity:group_content'migration_dependencies: optional: - upgrade_d7_node_og_blog - d7_user
Note the use of skip_creator
plugin to stop creating membership if current row user uid is the same as the group creator, to prevent duplicated members. The code of src/Plugin/migrate/process/SkipCreator.php
is:
namespace Drupal\og_migrate_group\Plugin\migrate\process;use Drupal\migrate\MigrateExecutableInterface;use Drupal\migrate\ProcessPluginBase;use Drupal\migrate\MigrateException;use Drupal\migrate\MigrateSkipRowException;use Drupal\migrate\MigrateSkipProcessException;use Drupal\migrate\Row;/** * Skip if user_key as uid is the creator of group with gid as in gid_key. * * @MigrateProcessPlugin( * id = "skip_creator" * ) * * To skip if creator use the following: * * @code * field_creator: * plugin: skip_creator * user_key: uid * gid_key: gid * method: row * @endcode * */class SkipCreator extends ProcessPluginBase { /** * Stops processing the current property when user_key as uid is the creator of group with gid as in gid_key. * * @param mixed $value * The input value. * @param \Drupal\migrate\MigrateExecutableInterface $migrate_executable * The migration in which this process is being executed. * @param \Drupal\migrate\Row $row * The row from the source to process. * @param string $destination_property * The destination property currently worked on. This is only used together * with the $row above. * * @return mixed * The input value, $value, if it is not empty. * * @throws \Drupal\migrate\MigrateSkipProcessException * Thrown if the source property is not set and rest of the process should * be skipped. */ public function process($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { $group = \Drupal::entityTypeManager() ->getStorage('group') ->load($row->getSource()[$this->configuration["gid_key"]]); if (!$group) { throw new MigrateException('gid not found.'); } if ($group->getOwner()->id() == $row->getSource()[$this->configuration["user_key"]]) { throw new MigrateSkipRowException("Creator is the same uid, so it exists"); } return $value; } /** * Skips the current row when user_key as uid is the creator of group with gid as in gid_key. * * @param mixed $value * The input value. * @param \Drupal\migrate\MigrateExecutableInterface $migrate_executable * The migration in which this process is being executed. * @param \Drupal\migrate\Row $row * The row from the source to process. * @param string $destination_property * The destination property currently worked on. This is only used together * with the $row above. * * @return mixed * The input value, $value, if it is not empty. * * @throws \Drupal\migrate\MigrateSkipRowException * Thrown if the source property is not set and the row should be skipped, * records with STATUS_IGNORED status in the map. */ public function row($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { $group = \Drupal::entityTypeManager() ->getStorage('group') ->load($row->getSource()[$this->configuration["gid_key"]]); if (!$group) { throw new MigrateException('gid not found.'); } if ($group->getOwner()->id() == $row->getSource()[$this->configuration["user_key"]]) { throw new MigrateSkipRowException("Creator is the same uid, so it exists"); } return $value; }}
So now we have groups and their members. Now we can start thinking about the content.
Content
Here instead of doing everything at once (create nodes and add as content to groups) as in https://thinktandem.io/blog/2018/03/30/migrating-drupal-7-organic-groups-to-drupal-8-group/ , it will be based in a previously migrated node set, as it was did in previous step where from users it created group memberships.
Let's create the group content first setting that content as available in group type settings/content
, in this example blog
group available content will be proposta
, blog_post
(as blog entry), podcast_episode
(as audio entry). Despite we could create a new plugin which transforms from nid
to title
or uid
, may be easier to do it in source view group_content csv, so including these properties. Relevant view used to obtain source data will be:
Relations:
- OG membership: Content from OG membership: 'og_membership_related_node'; REQUIRED
- Entity Reference : Referenced Entity: 'og_group_ref_target_id';
Fields:
- id, label: 'og_membership_id'
- (relationship: og_membership_related_node_1)->nid , label:content_nid
- (relationship: og_membership_related_node_1)->type, label: content_bundle
- (relationship: og_membership_related_node_1)->uid, label: content_uid
- (relationship: og_membership_related_node_1)->title, label: content_title
- gid, label: 'gid'
- (relationship:og_group_ref_target_id)->type, label: group_type
(With the filters and sorts adapted to your case, to prevent listing non-migratable content)
This will output
"og_membership_id","content_nid","content_bundle","content_uid","content_title","gid","group_type""13397","59546","blog","438","Good news","22477","bloc""13426","77","podcast","6","El sistema universitari","22447","bloc""13427","86","proposta","16","Estrobaca","22455","bloc""13428","87","blog","8","ssl","22449","bloc"
Here the yml that will process this csv:
langcode: enstatus: truedependencies: { }id: upgrade_d7_node_og_blog_contentclass: nullfield_plugin_method: nullcck_plugin_method: nullmigration_tags: - CSVmigration_group: insert_grouplabel: 'blog Group Content Migration from CSV'source: plugin: csv # Must be obtained from source with some exporter (e.g. views data export) and placed in this path path: 'public://imports/group-content-export.csv' header_row_count: 1 ids: - og_membership_id column_names: - og_membership_id: Identifier - content_nid: NodeID - content_bundle: bundle - content_uid: uid - content_title: title - gid: group - group_type: GroupType delimiter: ',' enclosure: '"'process: label: # TO show the current item to debug #- # plugin: callback # callable: var_dump # source: content_title - plugin: get source: content_title type: plugin: static_map source: content_bundle bypass: false # Customize this to match your values and group content plugins map: proposta: blog-group_node-proposta blog: blog-group_node-blog_post podcast: blog-group_node-podcast_episode default_value: blog-group_node-blog_post gid: gid entity_id: # To skip if element is not found (may be merged during translation # nodes migration fixes) - plugin: get source: content_nid - plugin: entity_lookup entity_type: node value_key: nid - plugin: skip_on_empty method: row uid: content_uiddestination: plugin: 'entity:group_content'migration_dependencies: null
And that's all, after that Groups, Memberships and Content will be migrated.
Bonus track - rename machine name of a bundle when migration has been already executed.
First of all we create the bundle giving the desired name, and with the module field_tools we can make the clone of view_modes first and after that the fields clones. Once it is identical, we must rollback the migration of the content a given bundle: drush migrate:rollback d7_node_complete:podcast
and then copy from web/sites/default/files/config__secret_directory_that _must_be_uniquely_and_obscured_from_public_found_at_settings/sync/migrate_plus.migration.upgrade_d7_node_complete_podcast.yml
to a dedicated custom module in its migrations
directory, then modify file setting the destination to desired bundle machine_name, also must be changed the id and filename according to what you are doing(in this case we set to rename_d7_node_complete_podcast
), then run drush migrate:rollback d7_node_complete:podcast --verbose && drush mim rename_d7_node_complete_podcast --verbose
to import again the content that we want. Any configuration over it must be executed once again, for example url_redirects migrations.
Bonus track 2 - node feeds in D7 to feeds_feed entity in D8
First create the feed types and add the fields as it was defined in source nodes that were acting as importers, then edit add the migration configuration, two working samples are in migrations
:
- d7_node_complete_og_to_canal_blog_extern.yml
- d7_node_complete_og_to_canal_proposta_externa.yml
Both are based on the stock migration configurations that migrateplus has created and exported in config sync directory: upgrade_d7_node_complete_[bundle_of_source]
. You can edit to create the feed entity instead of a node. Be careful to rollback previous imported bundle nodes migrations that match (in example it was feed
and proposta_feed
bundles). Flush plugins cache, and after that drush mim d7_node_complete_node_to_[bundle_of_source]
will do the magic. This is the first step, Now you have two options because feed source url is not easily picked.
One way and recommended as it can be automatizated, is using a new source plugin to fix the feed urls, look at commit this is a new plugin that will lookup the feed in source d7 feed table and "magically" will provide the url using some config as this:
langcode: en
status: true
dependencies: { }
id: d7_feed_sources
class: Drupal\migrate\Plugin\Migration
field_plugin_method: null
cck_plugin_method: null
migration_tags:
- 'Drupal 7 feeds'
- 'resync'
migration_group: resync
label: 'Drupal 7 Feeds sources'
source:
plugin: d7_feed
process:
fid:
-
plugin: entity_lookup
source: title
value_key: title
entity_type: feeds_feed
# -
# plugin: callback
# callable: var_dump
source: source
destination:
plugin: 'entity:feeds_feed'
migration_dependencies:
optional:
- d7_node_complete_og_to_canal_blog_extern
- d7_node_complete_og_to_canal_proposta_externa
The other way is using csv in a manual way, first instead of struggling too much I end up creating a view in source that outputs to csv these fields:
- Content: title (title)
- Feeds source: Source (source)
- Content: type (bundle)
- Content: Nid (nid)
Which csv I copied to public://imports/export-feeds.csv
And created the auxiliary migration configuration to import from csv:
langcode: enstatus: truedependencies: { }id: fix_feeds_sourcesclass: Drupal\migrate\Plugin\Migrationfield_plugin_method: nullcck_plugin_method: nullmigration_tags: - 'Drupal 7 feeds'migration_group: fix_feedslabel: 'Fix Feeds sources'source: plugin: csv path: 'public://imports/export-feeds.csv' header_row_count: 1 ids: - nid column_names: - title: title - source: Source - bundle: bundle - nid: Identifier delimiter: ',' enclosure: '"'process: fid: - plugin: entity_lookup source: title value_key: title entity_type: feeds_feed# -# plugin: callback# callable: var_dump source: sourcedestination: plugin: 'entity:feeds_feed'migration_dependencies: optional: - d7_node_complete_og_to_canal_blog_extern - d7_node_complete_og_to_canal_proposta_externa
then drush mim fix_feeds_sources
again. Then create the feed mappings as it was done in source(or modify as you want). And start importing