JSON Migrate Drupal 8 With Taxonomy Example

November 27th, 2017  | Categories: Drupal, Drupal 8, Drush 8, Example

I FINALLY got JSON to import into Drupal 8!

Set up a content type like so:

content type with body, origin link which is just a text field, categories as taxonomy entity reference

The Test Categories 0 is a taxonomy reference:

categories_0 is the machine name of the taxonomy

Here is my JSON (from this website actually; using an RSS to JSON converter; HTML escaped; just demo code anyway):

{

    "status": "ok",
    "feed": {
        "url": "http://www.longlivethemonkey.com/syntaxnotes/?feed=rss2",
        "title": "Syntax Notes @ LongLiveTheMonkey.com",
        "link": "http://www.longlivethemonkey.com/syntaxnotes",
        "author": "",
        "description": "notes, how to's, tutorials, problem solving in COMPUTER SCIENCE",
        "image": ""
    },
    "items": [
        {
            "title": "Default Drupal 8 Module",
            "pubDate": "2016-09-26 19:13:01",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=262",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=262",
            "author": "admin",
            "thumbnail": "",
            "description": "I may need this for later for module development with Drupal 8. It took me a while to figure out how to do it so I thought I’d post it. Start of Flex Carousel module Put this in the /modules/ folder. It is just a blue print to start a Drupal 8 module. It also […]",
            "content": " I may need this for later for module development with Drupal 8. It took me a while to figure out how to do it so I thought I’d post it. <a href=\"http://www.longlivethemonkey.com/syntaxnotes/wp-content/uploads/2016/09/flex_carousel.zip\">Start of Flex Carousel module</a> Put this in the /modules/ folder. It is just a blue print to start a Drupal 8 module. It also contains block region functionality for the module. I’m still very much learning Drupal 8 in the free time I have at work. Interesting stuff so far but I much prefer Drupal 7 since I’m a lot more used to it and it’s more straightforward, in my opinion.",
            "categories": [
                "Drupal 8",
                "Things To Remember"
            ]
        },
        {
            "title": "SharePoint 2013 Notes I",
            "pubDate": "2015-02-26 21:38:13",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=243",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=243",
            "author": "admin",
            "thumbnail": "",
            "description": "Well here’s another post of nifty things to remember. I’m working with SharePoint 2013 (YUCK!) and I ran into a few idiotic idiosyncrasies about it. Here’s a quick post. Classic SharePoint Breadcrumbs SharePoint breadcrumbs are hidden in 2013. They’re not included in the default master page… WTF? And when you enable them in your master […]",
            "content": "Well here’s another post of nifty things to remember. I’m working with SharePoint 2013 (YUCK!) and I ran into a few idiotic idiosyncrasies about it. Here’s a quick post. Classic SharePoint Breadcrumbs SharePoint breadcrumbs are hidden in 2013. They’re not included in the default master page… WTF? And when you enable them in your master page (if you can even figure out how to modify the master page), they’re really stupid. They’re a drop down. First let’s talk about the master page for a sentence or two… The new master pages are only modifiable via their HTML counterpart. So make a copy of the default seattle.html “master page” and that will create another one for you. Rename it as you like and it’s .master will be renamed too. Really stupid, huh? So you’re just supposed to remove the display: none on the  .ms-breadcrumb-dropdownBox  div and that brings back the “breadcrumb” but it’s not your typical breadcrumb. It’s a dropdown breadcrumb which sucks ass. Want a normal breadcrumb? Here’s the code… <div class=\"less_crappy_sharepoint_breadcrumb\"> <!--SPM:<asp:sitemappath     runat=\"server\"     sitemapprovider=\"SPContentMapProvider\"     rendercurrentnodeaslink=\"true\"     nodestyle-cssclass=\"breadcrumbNode\"     currentnodestyle-cssclass=\"breadcrumbCurrentNode\"     rootnodestyle-cssclass=\"breadcrumbRootNode\"     SkipLinkText=\"\"/>--> </div> Add this right below the  .ms-breadcrumb-top  div is what I did. I also removed the crappy suck ass breadcrumb since it’s useless to display it then hide it with CSS. Useless! Styles In The Masterpage Apparently since everything is done with this .html file, any ASP code or any other code in style tags (probably script as well) has to be “commented out.” Use the following if you want to put styles in your master page. <!--MS:<style type=\"text/css\">--> .ms-core-listMenu-horizontalBox li.static > a { display: none !important; } <!--ME:</style>--> Lame, huh? Oh, and watch out. For some reason SharePoint 2013 don’t do the > anymore so you’re probably going to have to put things like that in the style sheet… (always a good idea anyway). Hiding The First Top Menu Item How stupid is this? By default your site name is next to the logo… like you need it twice or something. Want it gone? Tough cookies. You’ll have to hide it with CSS. Here’s your CSS: .ms-core-listMenu-horizontalBox li.static > a { display: none !important; } .ms-core-listMenu-horizontalBox li.static > ul a { display: block !important; } Stupid! I hate SharePoint and I hate SharePoint 2013! How can anyone think this CMS is good. It’s a heavy footprint and look at all the work you have to do for basic things! I might update this post with more stuff you shouldn’t have to do to get SharePoint to look and function the way websites should OOTB. #SharePointGetALife #SharePointYouSuck #SharePointWTF",
            "categories": [
                "MS SharePoint 2013",
                "NIFTY TRICKS",
                "Things To Remember"
            ]
        },
        {
            "title": "Drupal 7 Notes I",
            "pubDate": "2015-02-13 20:14:11",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=240",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=240",
            "author": "admin",
            "thumbnail": "",
            "description": "Well this post isn’t exactly a PROBLEM/SOLUTION post but more of an entry to put the few things I need to remember on Drupal 7. Here are three things I should remember for the next time I need help with it. I’m naming it I just in case I want more than one. Limit Site […]",
            "content": " Well this post isn’t exactly a PROBLEM/SOLUTION post but more of an entry to put the few things I need to remember on Drupal 7. Here are three things I should remember for the next time I need help with it. I’m naming it I just in case I want more than one. <h2>Limit Site Search Results By Content Type</h2> If you ever need to exclude some content types from the site search, do the following: <ol> <li>Install and enable the <a title=\"Drupal - Search Configuration Module\" href=\"https://www.drupal.org/project/search_config\" target=\"_blank\">search_config<a> module.</li> <li>Go to #overlay=admin/people/permissions%23module-search_config and at the bottom you can limit anonymous users by content type.</li> </ol> <h2>Remove Authoring Information From Search Results</h2> On the search results page, by default, the authoring information is displayed. This kind of stuff should be turned off my default (in my opinion) since sites usually do not expose this information to the user. Only blogs or news articles have author info so it should be turned off for site search, on pages, etc but that’s a different vent. Anyway to turn this information off, do the following: <ol> <li>In /modules/search copy the search-result.tpl.php and paste it into the theme root (like /sites/all/themes/your_theme/ )</li> <li>Remove the authoring information code.</li> <li>Save the page.</li> <li>Clear the Drupal cache.</li> </ol> <h2>Search Title & Content Body In View Block</h2> <ol> <li>In the view, select the Tontent: Title and Content:Body under the Fields</li> <li>If you do not want the Body (or any other thing exposed in the list) select the “Exclude from display” in the Configure field: Content:Body.</li> <li>Under Filter Criteria, choose the “Global: Combine fields filter”.</li> <li>Check the “Expose this filter to visitors, to allow them to change it”, “Single filter”.</li> <li>For the Operator, choose “Contains” and select both/all fields under “Choose fields to combine for filtering”.</li> </ol> <h2>Permissions On The /sites/default/files Folder</h2> I struggled with permissions issues for like two days. Between going back and forth with IT on what’s allowed and what isn’t allowed. Blogs everywhere say that the files directory should be 777. But that’s access to everything for everybody. Ours has this access for the directory: drwxrwsr-x Every folder underneath this one should be granted the same privileges. By the way, this is on a UNIX box. ",
            "categories": [
                "Drupal",
                "NIFTY TRICKS",
                "Things To Remember"
            ]
        },
        {
            "title": "Add HTML5 Shiv To Drupal",
            "pubDate": "2014-07-01 15:26:24",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=223",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=223",
            "author": "admin",
            "thumbnail": "",
            "description": "PROBLEM I’m using HTML5 in my theme. I want older browsers (ie, IE) to recognize some of the new tags such as header, footer, etc. Google has a shiv out there you can download. I use this on like all my sites. You can link to it directly but I want the print shiv as […]",
            "content": " PROBLEM<br> I’m using HTML5 in my theme. I want older browsers (ie, IE) to recognize some of the new tags such as header, footer, etc. <a title=\"Google - HTML5 Shiv\" href=\"https://code.google.com/p/html5shiv/\" target=\"_blank\">Google has a shiv</a> out there you can download. I use this on like all my sites. You can link to it directly but I want the print shiv as well… I’m not sure where they keep that so you can directly link to it. I’m a novice at Drupal 7 and they’ve changed a bunch since Drupal 6. The JavaScript calls are different. Also these JS files are a little unique as I want to put them in an less than IE9 comment. SOLUTION<br> After playing around with getting the URL to the theme, having that not work exactly since I’m working on a local environment, I got the function calls I needed to link to the /javascript/ folder I created in the theme where the shivs will go. I modified the html.tpl.php file which is in the themes folder. I added the following two lines of code which go to the two shivs: <!--[if lt IE 9]><script src=\"<?= base_path().path_to_theme(); ?>/javascript/html5shiv.js\"></script><![endif]--> <!--[if lt IE 9]><script src=\"<?= base_path().path_to_theme(); ?>/javascript/html5shiv-printshiv.js\"></script><![endif]--> If there’s another way, I’d like to know. I’m sure there’s a module out there you can download to get the HTML5 support like a shiv, but I think linking to the shivs is a lot easier. ",
            "categories": [
                "Drupal",
                "MS IE"
            ]
        },
        {
            "title": "Bottom Border Not Showing Up In Nested List In IE",
            "pubDate": "2012-06-27 15:12:01",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=178",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=178",
            "author": "admin",
            "thumbnail": "",
            "description": "PROBLEM In IE the bottom border of a div with many nested lists isn’t showing up. It works fine in FF. All the other borders are fine and visible. As you can see below, the bottom border does not show up in IE: The only CSS for the main container is a border. However, the […]",
            "content": " PROBLEM In IE the bottom border of a div with many nested lists isn’t showing up. It works fine in FF. All the other borders are fine and visible. As you can see below, the bottom border does not show up in IE:<br><a href=\"http://www.longlivethemonkey.com/syntaxnotes/wp-content/uploads/2012/06/2012-06-27_110417.png\"><img class=\"aligncenter size-medium wp-image-179\" title=\"Bottom Border Not Showing Up In IE\" src=\"http://www.longlivethemonkey.com/syntaxnotes/wp-content/uploads/2012/06/2012-06-27_110417-106x300.png\" alt=\"\"><a> The only CSS for the main container is a border. However, the funky thing about this issue is that when I only have one list, the border shows up (just imagine the list stopping at “Resources and Publications” in the above example). That would only be one list. Searching the web didn’t yield any fix. When you google this issue, they usually suggest adding hasLayout and position: relative to every li. That didn’t work for me. SOLUTION<br> By adding a min-height to the container div did the trick for me. This was the same div that had the border CSS. I added min-height: 10px and the border now works in IE. ",
            "categories": [
                "CSS",
                "MS IE"
            ]
        },
        {
            "title": "Two Column List With CSS",
            "pubDate": "2012-03-07 14:46:36",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=173",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=173",
            "author": "admin",
            "thumbnail": "",
            "description": "PROBLEM I wanted to do a list of items in two columns with a border around the list. I didn’t want to use two floated divs in order to get the look I wanted (nor tables). Also, I didn’t want to break up the list. I wanted the HTML to be simple. I started out […]",
            "content": " PROBLEM I wanted to do a list of items in two columns with a border around the list. I didn’t want to use two floated divs in order to get the look I wanted (nor tables). Also, I didn’t want to break up the list. I wanted the HTML to be simple. I started out with some old code I found which used floats and widths on each li. However after the list I needed to do a clear so the border would go around the entire list. This clear caused problems since it cleared my other floated divs (for page layout). Here’s the HTML I wanted to use: <blockquote> <div class=”summary_container”><br> <ul class=”two_column_ul”><br> <li>Overview</li><br> <li><a title=”Training & Education” href=””>Training & Education</a></li><br> <li><a title=”Staff” href=””>Staff</a></li><br> <li><a title=”Collaborators” href=””>Collaborators</a></li><br> <li><a title=”Research Areas” href=””>Research Areas</a></li><br> <li><a title=”Scientific Articles” href=””>Scientific Articles</a></li><br> <li><a title=”Scientific Resources” href=””>Scientific Resources</a></li><br> <li><a title=”Image Library” href=””>Image Library</a></li><br> </ul><br> <!– this is where the clear would go if necessary <div class=”clear”></div> –><br> </div> </blockquote> SOLUTION<br> After attempts at floating and clearing the float, I eventually gave up. I tried to do an inline-block on the li, which worked great in FireFox but IE choked. The CSS I finally used was the following: <blockquote>  div.summary_container {<br> border: 1px solid #DEE6F3;<br> padding: 5px 5px 15px;<br> } div.summary_container ul.two_column_ul {<br> margin: 0 0 0 20px;<br> padding: 0;<br> list-style: none;<br> } div.summary_container ul.two_column_ul li {<br> display: -moz-inline-stack;<br> display: inline-block;<br> zoom: 1;<br> *display: inline;<br> margin: 0;<br> padding: 6px 0 0 14px;<br> width: 220px;<br> }  </blockquote> I hope this can help someone else. ",
            "categories": [
                "CSS",
                "MS IE",
                "NIFTY TRICKS",
                "XHTML"
            ]
        },
        {
            "title": "Gap At Bottom Of Div With Image In IE",
            "pubDate": "2012-01-27 15:06:24",
            "link": "http://www.longlivethemonkey.com/syntaxnotes/?p=167",
            "guid": "http://www.longlivethemonkey.com/syntaxnotes/?p=167",
            "author": "admin",
            "thumbnail": "",
            "description": "PROBLEM Here’s another one of those weird IE bugs. There is a gap at the bottom of the div that has an image in it. The image may or may not be linked. In my case, it was a banner that has an image that is using an (ew) image map for links. There is […]",
            "content": " PROBLEM Here’s another one of those weird IE bugs. There is a gap at the bottom of the div that has an image in it. The image may or may not be linked. In my case, it was a banner that has an image that is using an (ew) image map for links. There is a strange gap at the bottom between the two divs. Only in IE, of course. <a href=\"http://www.longlivethemonkey.com/syntaxnotes/wp-content/uploads/2012/01/2012-01-27_100315.png\"><img class=\"aligncenter size-full wp-image-169\" title=\"2012-01-27_100315\" src=\"http://www.longlivethemonkey.com/syntaxnotes/wp-content/uploads/2012/01/2012-01-27_100315.png\" alt=\"Small two pixel gap between divs\"><a> There isn’t anything in my CSS that would cause this, of course. SOLUTION<br> Since IE has a problem… well, let’s face it, it has many problems… It thinks there is to be text in that space. Or whatever. To get rid of the gap add font-size: 0; to the div which contains the image. The gap is closed. I need to remember this fix. ",
            "categories": [
                "CSS",
                "MS IE"
            ]
        }
    ]
}

All around the web I find tutorials on how to import JSON into Drupal 8 using migrate but very few actually have the JSON so I can look to see what’s actually happening in it. That’s why I chose to put all that demo/crappy JSON above. This JSON should be placed somewhere so that your Drupal site can access it.

If you have not already done so make sure to install and activate the migrate_plus and migrate_tools modules for Drupal 8.

Now here’s what you’ve all been waiting for… the YML part! Go to /admin/config/development/configuration/single/import in Drupal 8.

For Configuration type put in Migration.

Here’s my YML:

id: migration_try_6
label: 'Import from JSON 6'
migration_dependencies: {}

source:
  plugin: url
  urls: 'http://localhost/temp/tempjson.json'
  data_fetcher_plugin: http
  data_parser_plugin: json

  item_selector: items/
  fields:
    -
      name: sn_title
      label: 'Title'
      selector: title
    -
      name: sn_pubDate
      label: 'Pub Date'
      selector: pubDate
    -
      name: sn_link
      label: 'Link'
      selector: link
    -
      name: sn_guid
      label: 'GUID'
      selector: guid
    -
      name: sn_description
      label: 'Description'
      selector: description
    -
      name: sn_content
      label: 'Body'
      selector: content
    -
      name: sn_categories
      label: 'Categories'
      selector: categories
  ids:
    sn_guid:
      type: string
process:
  type:
    plugin: default_value
    default_value: tester_mc_testy
  title: sn_title
  field_origin_link_0: sn_link
  body: sn_content
  created:
    plugin: format_date
    from_format: 'Y-m-d H:i:s'
    to_format: U
    source: sn_pubDate
  status:
    plugin: default_value
    default_value: 1
    
  # used from http://www.amitgoyal.in/2016/11/json-d8-migration-process-plugins.html
  field_test_categories_0:
    plugin: entity_generate
    value_key: name
    bundle_key: vid
    entity_type: taxonomy_term
    source: sn_categories
    bundle: categories_0
    
  sticky:
    plugin: default_value
    default_value: 0
  uid:
    plugin: default_value
    default_value: 0
destination:
  plugin: 'entity:node'

Hit Import then yes or whatever. Please note that this DOES NOT run the migration for you! That’s handled later using Drush!

Trying to explain this can be rather tricky so I’d rather just leave it as a reference and try to explain a little bit. The urls is what the URL to your JSON file is. Anything that’s “sn_…” is just a placeholder. You can name it anything so long as it’s the same as what’s in the source in the process section.

The taxonomy works! So it will create taxonomy that isn’t there and it will reference taxonomy that already is in there. That was the trickiest part. I thought I’d have to write a class/method to do it but Drupal 8 entity_generate does it automatically which is really nice.

Now to run this baby go into Drush (your ghetto command line) and do:

$ drush migrate-import migration_try_6

Or drush… or you can do a drush ms to see if it’s there.

Here’s what mine looked like:

the command and it showing 7 created, 0 updated, 0 failed, 0 ignoredNow unfortunately the migration does not show up under /admin/structure/migrate/manage/default/migrations . I’m not sure why or how to get it to show up there. I’m thinking it’s another bug with migrate.

To roll the migration back just do drush migrate-rollback migration_try_6 which will delete all the content (but NOT the taxonomy).

Migrate in Drupal 8 is still VERY NEW and VERY BUGGY. They haven’t worked out all of the kinks and to my experience it constantly changes.

This code works on Drupal 8.4.2 using Drush 8.1.15 . A lot of the migrate drush commands do not work for me such as drush migrate-deregister. I don’t know why.

Also if you ever have an issue with your YML, even though it “imported” fine with the GUI (single import step) it still may have issues with Drush. For example I cannot do drush ms without getting errors with my first import. Since drush migrate-deregister doesn’t work for me I have no way to remove it. It also does not show up in Drupal so I can’t remove it that way either… lots of bugs!

Anyway I hope this helps someone else as I put in everything I can think of as an example.

No comments yet.

TOP