Create a select menu from subfolders

If you want to run some code on a directory, but maybe not always the same directory, then generating a menu can be very handy.
# The menu we are making looks like this
[me@parentfolder]$ directoryPicker $pwd
1) Subfolder_1
2) Subfolder_2
3) Subfolder_3
#? 
To make a generic menu, call select with an array, and it displays the choices. You can get complicated and have different return or output values for each of the choices but for this example, we are just setting a variable, DIRECTORY, which is returned to the calling function.
# The select function portion of the example
select dir in "${dirs[@]}"; do
  # This will be run only on the user's choice, the function itself 
  # handles displaying the array values as a menu
  DIRECTORY="${PARENT}/${dir}"
  break;
done
You can declare an array with values for each item under a directory by using a subshell to call ls.
# make an array using a directory listing
declare -a dirs=($(ls -d *))

# Want to check what is in here?  Echo all the entries with [*]
echo ${dirs[*]}
But you probably want to do a little filtering to remove files, since passing them to your calling function won’t do much and it makes the menu less readable.

The basics of looping through a bash array probably look familiar, but for the iteration note that ${array[@]} will give you each array value in its own variable, and ${array[*]} will return them all as a single variable.
# a basic array loop
for dir in "${dirs[@]}"; do
  echo ${dir}
done
To remove an item from an array, you want to unset that index, so you’re going to give this loop an index and increment the index using let.
index=0
for dir in "${dirs[@]}"; do
  if [ ! -d "${PARENT}/${dir}" ]; then
    # unset an array element by referencing the variable _name_, 
    # not using the variable. But the key will be the variable _value_ 
    # so you still have a $ in there.
    # e.g. array[1]
    unset 'dirs[$index]'
  fi
    
  # Incrementing a variable either requires arithmetic expansion with 
  # double parenthesis, or using let
  let "index++"
done
All put together, you get this handy function that takes a parent directory and returns a menu to choose only subdirectories.
# Fetch a list of directories under the passed one
# @param[1] parent directory
# @returns DIRECTORY, directory path
function directoryPicker {
  # $1 = Parent Directory
  PARENT=$1
  cd $PARENT

  # make an array using a directory listing
  declare -a dirs=($(ls -d *))
    
  # We want to display only folders as options, 
  # so remove non-directories
  index=0
  for dir in "${dirs[@]}"; do
    if [ ! -d "${PARENT}/${dir}" ]; then
      # unset an array element by referencing the variable name, 
      # not using the variable. But the key will be the variable value
      # e.g. array[1]
      unset 'dirs[$index]'
    fi
    
    # Incrementing a variable either requires arithmetic expansion with 
    # double parenthesis, or using let
    let "index++"
  done

  # Then display the array as a numbered choice list
  select dir in "${dirs[@]}"; do
    DIRECTORY="${PARENT}/${dir}"
    break;
  done
}
To put this select menu into use, you can call this function from inside another, passing it a parent directory. This example will fetch all the updates for all the git repos in a subfolder. I have my repos checked out into groups based on the organization or user that owns them, or based on a theme they have in common. So there are folders for my employer, my personal stuff, friends, and themed project topics like “WordPress” or “cryptocurrency”. So passing my updateRepos function a menu lets me update only a specific set of repos at once.
# Update all the repos under a parent folder
function updateRepos {
  # Get a list of subfolders, using our chooser to show a list
  directoryPicker "/Users/$(whoami)/Developer/github.com"
  REPOBASE=$DIRECTORY

  # Like we did in the chooser, create an array of subfolders
  cd $REPOBASE
  declare -a repos=($(ls -d *))

  # for each repo, fetch/pull latest
  for repo in "${repos[@]}"; do
    # Only bother with directories that have a .git subfolder
    if [[ -d $REPOBASE/${repo} && -d $REPOBASE/${repo}/.git ]]; then
      echo "Checking ${repo} for updates"
      cd $REPOBASE/${repo}
      git fetch
      git pull
    fi
  done
}

Verify your site(s) with Google Webmaster Tools

You can load simple synthetic files in nginx just by returning a value.

Drop this snippet into the server block in your host configs.

Even better, make an include for all your virtual hosts and put this in there!

# Google Webmaster tools Verification file
# You can get your verification file in the console when you add or verify a site
# https://www.google.com/webmasters/tools/home?hl=en&authuser=0
location = /google123456RANDOMNUMBER.html {
    add_header Content-Type text/html;
    return 200 'google-site-verification: google123456RANDOMNUMBER.html';
}

Test your error pages

Set up URLs to test a variety of error pages, and how your WordPress site will handle warnings and notices as well. This doesn’t do an awful lot for a simple WordPress site, but if you are running through a CDN, updating your logging, or sending data to analytics it can give you a tool to intentionally send error states for testing those tools.

<?php
/*
Plugin Name: Throw Stuff
Description: Generate errors through wordpress to verify debugging
Version: 0.1
Author: Katherine Semel
*/

class ThrowStuff {

    function __construct() {
        add_action( 'template_redirect', array( $this, 'set_endpoints' ) );
    }

    function set_endpoints() {

        // Give this a status code and it will give it back.
        // Make sure it's a number!
        if ( strpos( $_SERVER['REQUEST_URI'], 'test/httpcode' ) == 1 ) {
            $path = parse_url( $_SERVER['REQUEST_URI'], PHP_URL_PATH );
            $pathFragments = explode( '/', $path );
            $request_code = end( $pathFragments );

            if ( is_int( intval( $request_code ) ) ) {
                status_header( intval( $request_code ) );
            } else {
                status_header( 200 );
            }
            echo $this->error_body( intval( $request_code ) );
            die();
        }

        if ( strpos( $_SERVER['REQUEST_URI'], 'test/notice' ) == 1 ) {
            // E_USER_NOTICE - this is the equivalent of E_NOTICE and
            // issues a notice message. 
            // Execution of the script is not halted.

            trigger_error( 'Test Generic E_USER_NOTICE', E_USER_NOTICE );
            status_header( 200 );
            echo $this->error_body( 'Test Error: Notice' );
            die();
        }

        if ( strpos( $_SERVER['REQUEST_URI'], 'test/warning' ) == 1 ) {
            // E_USER_WARNING - this is the equivalent of E_WARNING and 
            // issues a warning message. 
            // Execution of the script is not halted.

            trigger_error( 'Test Generic E_USER_WARNING', E_USER_WARNING );
            status_header( 200 );
            echo $this->error_body( 'Test Error: Warning' );
            die();
        }

        if ( strpos( $_SERVER['REQUEST_URI'], 'test/error' ) == 1 ) {
            // E_USER_ERROR - this is the equivalent of E_ERROR and 
            // indicates an error that can not be recovered from. 
            // Execution of the script is halted at this point.

            trigger_error( 'Test Generic E_USER_ERROR', E_USER_ERROR );
            status_header( 500 );
            echo $this->error_body( 'Test Error: Error/Fatal' );
            die();
        }
    }

    function error_body( $error_code_or_message ) {
        ob_start();
        ?>
        <html>
            <head>
                <title>Response Code <?php echo strval( $error_code_or_message ); ?></title>
            </head>
            <body>

                <?php
                if ( is_int( $error_code_or_message ) ) {
                    echo "I am an error testing page for code $error_code_or_message";
                } else {
                    echo strval( $error_code_or_message );
                }
                ?>
            </body>
        </html>
        <?php
        $output = ob_get_clean();
        return $output;
    }
}

// Fuck it, let's test in production
// Make sure you understand what you are doing!
$ThrowStuff = new ThrowStuff();

Prompt for a timespan in shell scripts

If you want to capture user input in a shell script (I prefer bash! Do what you want, but this script is bash), you’re looking for the command read.

This version gives the user a default start date 30 days in the past and will adjust the end date to be 30 days after whatever they enter.

#!/bin/bash
# Prompt a user for start and end dates

# Initialize start date variable with a default that is *30 days* ago
start_date=$( date -v -30d +"%Y-%m-%d" )

# Prompt the user for the start date
echo "What start date should I use? (format: YYYY-mm-dd, default: $start_date)"
read start_date_in
if [ "$start_date_in" != "" ]; then
    # If the user entered a date, use it
    start_date=$start_date_in
fi

# Initialize the end date as *30 days* after the start date
end_date=$( date -j -u -f "%Y-%m-%d" -v +30d "${start_date}" +"%Y-%m-%d" )

# Prompt the user for the end date
echo "What end date should I use? (format: YYYY-mm-dd, default: $end_date)"
read end_date_in
if [ "$end_date_in" != "" ]; then
    # Again, if the user entered a date, use it
    end_date=$end_date_in
fi

# Display the date span so the user knows!
echo "Processing between $start_date and $end_date"

# If you need to, convert these dates to timestamp for computer-friendly biz.
start_date_timestamp=$( date -j -u -f "%Y-%m-%d" "${start_date}" +"%s" )
end_date_timestamp=$( date -j -u -f "%Y-%m-%d" "${end_date}" +"%s" )

Generate a list of urls from a sitemap

When I’m load testing a site, I like to get a list of urls to run against. There’s not much point in checking the home page constantly, let’s find some variety.

This script expects a sitemap or sitemap index, and will give you back a text file with urls.

There is a dependency on xmlstarlet, a command line program for dealing with XML files. If you’re using homebrew it is simple to install xmlstarlet with brew install xmlstarlet.

When using XML to deal with a sitemap, and the namespaced elements in one, bind the namespace to a prefix and prepend it to the name, like this

xmlstarlet sel -N x='http://www.sitemaps.org/schemas/sitemap/0.9'

Source: http://xmlstar.sourceforge.net/doc/UG/xmlstarlet-ug.html#idm47077139669232

function get_urls_from_sitemap {
    # $1 sitemap_index
    SITEMAP_INDEX=$1

    OUTPUT_FIlE=urls.txt
    # Reset the output file
    : > $OUTPUT_FIlE

    # We use the namespaced in a few places so plop it here
    XMLSCHEMA='http://www.sitemaps.org/schemas/sitemap/0.9'

    # Check we got an XML file first by checking the content type
    isXML=$(curl -sS -o sitemap_index.xml -w '%{content_type}' "$SITEMAP_INDEX")

    # If it is an XML file, let's go with it. 
        # We'll get errors if it isn't a sitemap anyway
    if [[ $isXML = *"text/xml"* ]]; then

        echo "Getting urls from index: $SITEMAP_INDEX"

        # Read the sitemap index
        xmlstarlet sel -N x=$XMLSCHEMA -t -v '//x:loc' -n <sitemap_index.xml > sitemaps.txt

        # Then loop through the results!
        exec 4< sitemaps.txt
        while read <&4 SITEMAP; do

            # Some of these are url encoded, just quietly fix that!
            SITEMAP_URL=$(echo "$SITEMAP" | sed "s/\&amp;/\&/g")

            # This is the same content type check from before
            isXML=$(curl -sS -o sitemap.txt -w '%{content_type}' $SITEMAP_URL)

            if [[ $isXML = *"text/xml"* ]]; then
                # If this is an XML file, get more urls from it!
                echo "Getting urls from sitemap: $SITEMAP_URL"
                xmlstarlet sel -N x=$XMLSCHEMA -t -v '//x:loc' -n <sitemap.txt >> $OUTPUT_FIlE
            else
                # Just add non XML to the urls file
                echo $SITEMAP_URL >> $OUTPUT_FIlE
            fi

            rm -f sitemap.txt
        done

        rm -f sitemaps.txt
    else
        echo "Yo, this isn't an XML file"
    fi

    rm -f sitemap_index.xml
}

On code quality

To do most anything with a website, there is a great deal of code that needs to be written. Some of it is very important, and you should put care into the planning and creation of those components, modules, layouts, and etc. Those pieces, if poorly done, will gather issues like snowflakes and you’ll find yourself constantly shoveling out just to make a little progress.

However, “etc” is also filled with those less-important unnamed things. The code surrounding all those important parts that isn’t so vital to plan out or get honed to a fine edge. It isn’t going to last long, or do much, or in some cases even be run twice. You can cobble together some parts to solve a bug, or repair some data, or find out how many times that one IP address tried to log into your site over a week.

If you need to write some shitty code to solve something, don’t beat yourself up over it. Writing shitty code gets the work done, and the practice will help you to identify the parts you do need to spend some extra time getting right.