Mission:
I’m interested in migrating all my WriteFreely posts to Hugo but have hit a wall with dealing with tags.
I exported a CSV file from WriteFreely. (CSV file data: id,slug,blog,url,created,title,body)
Wrote very basic bash script to convert CSV file to .MD files.
#! /bin/bash
#Script to convert WriteFreely CSV to Hugo MD files
#WriteFreely CSV columns
#id,slug,blog,url,created,title, body (column1-8)
while IFS="," read -r column1 column2 column3 column4 column5 column6 column7
do
echo "---" | tee "${column2}".md
echo "title: \"${column6}\"" | tee -a "${column2}".md
echo "date: $column5" | tee -a "${column2}".md
echo "draft: false" | tee -a "${column2}".md
echo "---" | tee -a "${column2}".md
echo " " | tee -a "${column2}".md
echo -e "$column7" | tee -a "${column2}".md
done < <(tail -n +2 data.csv)
This works to create .MD files for posts in Hugo. The only thing that is missing are the tags.
This is an example of a file that the script created.
---
title: "The Watermelon Boys by Ruqaya Izzidien"
date: 2019-02-19T12:17:29Z
draft: false
---
"Book Review
>The present is an arrogant time in which to live, always has been. Humans of the present look back at their people, land, and history, and whisper to themselves with glee,
[more review here]
My Rating: **★★★★★**
My 2019 Reading Challenge: Book #11
#TranslatedFiction #BookReview"
As you see there are tags at the end of each post #tag
They need to be inserted in the frontmatter for each post.
The top needs to be like this:
---
title: "The Watermelon Boys by Ruqaya Izzidien"
date: 2019-02-19T12:17:29Z
tags: ["TranslatedFiction", "BookReview"]
draft: false
---
Goal:
I’d like to automate adding the tags with another script, or modify my original script.
Any suggestions on the best/quickest/easiest way to do this? I have 300+ posts I need to do this for.
modify above script to add a dummy line in the frontmatter
tag: [ XXXX88 ]
use LibreCalc formula to split body of CSV from the right on new line (\n) and just grab the last line containing tags (#tag#tag2#tag3)
create CSV file with two columns (slug, tags)
create new script:
– load CSV
– put column2 (tags) into an array variable, format it to be: “tag”, “tag2”, “tag3”
– use sed(?) to replace XXXX88 in the $slug.md with the array data
Now I just need to figure out the code for all this, or if there is an easier way, please let me know
Ok final script. Hope it helps others who are moving from WriteFreely to Hugo.
convertWritefreely: converts the Writefreely CSV to Hugo MD files. Make sure you clean up the CSV file in a spreadsheet program first.
#! /bin/bash
#Script to convert WriteFreely CSV to Hugo MD files
#WriteFreely CSV columns
#id,slug,blog,url,created,title, body (column1-8)
while IFS="," read -r column1 column2 column3 column4 column5 column6 column7
do
echo "---" | tee "${column2}".md
echo "title: \"${column6}\"" | tee -a ${column2}.md
echo "date: $column5" | tee -a "${column2}".md
echo "tags: [XXXXXX]" | tee -a "${column2}".md
echo "categories: [\"blog import\"]" | tee -a "${column2}".md
echo "draft: false" | tee -a "${column2}".md
echo "---" | tee -a "${column2}".md
echo " " | tee -a "${column2}".md
echo -e "$column7" | tee -a "${column2}".md
done < <(tail -n +2 data_full_body.csv)
convertTags: grabs ‘tags’ from end of WriteFreely CSV and replaces the placeholder in the *.MD files. Again make sure you clean up CSV file before running.
#! /bin/bash
#Script to convert WriteFreely CSV tags and past into the Hugo MD files
#tags CSV columns
#id,slug,tags,body (column1-4)
newTags="temp"
#Read in the data form the CSV file
while IFS="," read -r column1 column2 column3 column4
do
echo "starting tag data: ${column3}"
#Replace # from tags with sed
newTags=$(echo "$column3" | sed -e "s/n#//" | sed -e "s/\(#\)\1*//g")
echo -e "finish tag data: ${newTags}"
read -a tagArray <<< $newTags
echo "tag array start: ${tagArray[@]}"
#Loop through array and add quotes
quotes="\""
#for ((i=0; i<"$length+1"; i++))
# do
# tagArray[i]=${quotes}${tagArray[i]}${quotes}
# done
tagsFixed=()
count=0
for i in "${tagArray[@]}"
do
echo $i
tagArray[$i]=${quotes}${i}${quotes}
echo "${tagArray[$i]}"
tagsFixed[$count]=${tagArray[$i]}
count=$(($count+1))
done
echo "tag array end: ${tagsFixed[@]}"
echo ""
echo "here will write these tags into the Hugo.md files"
for (( i = 0 ; i < ${#tagsFixed[@]} ; i++ )) do tagsFixed[$i]=${tagsFixed[$i]}","; done
tagsOutput=${tagsFixed[@]%}
echo "${tagsOutput}"
sed -i "s:XXXXXX:$tagsOutput:" $column2.md #column2 = slug
echo ""
done < <(tail -n +2 data_full_tags.csv)
You will have to modify these to work for your data.
There isn’t any error checking.
Also script fails with non-English characters.
Use at your own risk.
If you modify this or improve it, please share with the community, or even make a Github repo
EDIT:
there is some problem when Hugo tries to parse the files, there is
title: "“blah”
use sed to remove
find -type f -exec sed -i -e 's|title: ""|title: "|g' {} \;
also for some reason all the # signs in the posts are //
I see that you have moved to hugo. Happy that you found a good software for your blog.
You may write an article “hugo for beginners” , it’ll be great.
You can also add your tuto to the hugo website
Great tutorial.
I liked the idea of the cron job, but I was just wondering if it would be possible to have more than one backup (filename"date".tar) as it does not take a lot of space and having multiple backups may be helpful one day.
Another thing about hugo is organising the posts. Categories, tags, folder structure. I am still reading about it so I don’t screw it up.
There is also the comments integration and moderation, but I didn’t look for the best option. And maybe the analytics.
#!/bin/bash
now=$(date +"%F")
filename=arkadicloud_$now
echo "* ARKADI-CLOUD BLOG *"
echo "** DELETE BACKUP **"
find /home/yunohost.backup/archives/ -type f -iname 'arkadicloud_*' -mtime +7 -delete
echo "*** MAKE BACKUP ***"
sudo tar -cf /home/yunohost.backup/archives/$filename.tar /var/www/my_webapp/www/
echo "**** SYNC PAGES TO ARKADI-ONE WEB SERVER ****"
sudo rsync -ah --delete /home/yunohost.app/nextcloud/data/elias/files/Documents/arkadicloud/public/ /var/www/my_webapp/www/
echo "** FINISHED! **"
I was also almost paralyzed by the overwhelming choice of ways to organize my content in the beginning. Just learn about one thing at a time or think of a specific goal you want to achieve. Then, copy your whole website to another directory and try things out! Hugo server will immediately tell you if something is wrong.
I disabled comments in my config.yaml. I don’t like Discuss and all that jazz.
Categories are built in just add something like this to front matter: categories: ["cool posts", "not cool posts"]
Then if your theme has somewhere to display this, it will.
For folder structure I think you want to be looking at archetypes. I am not too clear on how that works either. I have a simple blog though, I don’t even use pictures! That may change now once I figure out how to use them in hugo.