Jenkins email notifications

The default Jenkins email notifications are almost unusable, they contain a complete output of the build log. How to set up custom notifications:

• Install this plugin: https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
• Configure a Jenkins job, scroll to the bottom and click "Add post-build action", then select "Editable email notification"
• Example email design:

Subject: [CI] Nightly "master" test results - Build #${BUILD_NUMBER} - ${BUILD_STATUS}

Body:

Nightly master tests

Test run #${BUILD_NUMBER} for "${PROJECT_DISPLAY_NAME}"

Links: (1) details of successes/failures, (2) full console output, (3) latest commits to master

Summary

Result: ${BUILD_STATUS} - ${TEST_COUNTS, var="fail"} failures

${TEST_COUNTS, var="pass"} out of ${TEST_COUNTS, var="total"} tests passed (${TEST_COUNTS, var="skip"} skipped).

Details

${FAILED_TESTS, showStack=false, showMessage=false}

Elasticsearch basics

Warning: Your Elasticsearch / ELK stack based logging solution may take a huge amount of disk space, and indexing of large amounts of data may also take so long that it can't keep up with the logs being generated.

In short, you need massive/cluster/cloud resources to support Elasticsearch.

Search:

curl '{endpoint}/_search?q=title:jones&size=5&pretty=true'

List indexes:

curl -s '{endpoint}/_cat/indices?v' | sort

Upload a template:

curl -X POST -H "Content-Type: application/json" -d @path/to/template.json 'http://elastic:changeme@localhost:9200/_template/testlog?pretty'

Add a document:

curl -X POST -H "Content-Type: application/json" -d '{ "timestamp": "2019-04-06T14:13:31", "message": "bar baz qux" }' http://elastic:changeme@localhost:9200/testlog/footype?pretty

Range query:

curl -X GET -H "Content-Type: application/json" -d '{ "query": { "range" : { "timestamp" : { "gte" : "2019-04-02T15:13:31", "lte" : "2019-05-09T14:13:31", "boost" : 2.0 } } } }' http://elastic:changeme@localhost:9200/testlog/_search?pretty

Git: diff branch against master

Compare feature branch changes with upstream:

git diff $( git merge-base master HEAD )

Thanks Bill!

Surprising behaviour of the perl debugger for wantarray()

When you are running a perl script in the debugger, do not expect wantarray() to return the same thing it would if not running in the debugger.

If you break execution at a breakpoint, wantarray() seems to return 1 when it wouldn't otherwise.

Show the full patch for a git merge commit

Use the undocumented -m and -p options to show the main commit ID.
Why are these not in the help!?

git show -m -p 25172c1

Specify git config temporarily for one command

Use -c after the git call, before any other options:

e.g. to override your core.pager="less -r" setting:

    git -c 'core.pager=cat' log -n 3 --abbrev-commit

(source)

How to apply a named stash in git

git stash apply stash^{/foo1}

where foo1 is the full name of your stash (keep names short, with no spaces).

(source)

How to find who/what is changing a file in linux

A list:

• inotifywait - simple, attended
• auditctl - powerful, old school
• file system monitoring - more involved, more complete

See https://pinboard.in/u:random/t:audit

Reasons to use "git pull --rebase" by default

Rebase is a really sharp knife. Sometimes a really sharp knife is what you need, but it's also possible to accidentally injure yourself.

(thanks to Bill Blunn for the imagery).

Reasons to use "git pull --rebase" and not "git pull" (which merges by default):

• Prevents massive weird conflicting-with-yourself problems if anyone rebases that branch later, e.g. just before merging to master
• Keeps all the commits together in a bunch at the top of the tree so you can easily identify them visually
• Allows "git diff master..HEAD" to work without including unrelated changes

Reasons not to use "git pull --rebase" and stick with "git pull" (using merges):

• It's the default, so most people will use it, and their merges will clash with your rebases

Comparison table:

issue	git pull (default: merge)	git pull --rebase
git log, in a feature branch	pro: you always see when master was merged con: your commits will be interleaved with others commits (less of a problem for short-lived branches)	pro: root of feature branch is transparently moved to head of master pro: all your commits are kept bunched together con: you don't know when master was merged con: you have to notify everyone downstream
diff feature branch against master	con: it's more difficult to diff against master	pro: "git diff master" just works
mixing rebase and merge	con: you can't rebase without weird conflicts	con: you can't merge downstream without weird conflicts (not sure about this one)

(thank you tablesgenerator.com)

ssh debug3: Incorrect RSA1 identifier debug3: Could not load ".ssh/id_rsa" as a RSA1 public key

When testing ssh with -vvv, you see this in the log:

    debug3: Incorrect RSA1 identifier
    debug3: Could not load ".ssh/id_rsa" as a RSA1 public key

This is not an error.
This is not the problem you're looking for.
Your problem is something else.

(source)

See also another ssh non-error.

How to test CGI::Prototype

Why, use CGI::Prototype::Mecha of course.

debug2: key_type_from_name: unknown key type '-----BEGIN'

SSH debug output can be quite misleading.
When you see the following output, it does NOT indicate a problem.

debug2: key_type_from_name: unknown key type '-----BEGIN'
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
...
debug2: key_type_from_name: unknown key type '-----END'
debug3: key_read: missing keytype
debug1: identity file /home/foo/.ssh/id_rsa type 1

The final line tells you that the key was read successfully.

(source)

See also another ssh non-error.

Whitespace woes

One very annoying aspect of software development in a team is whitespace. Many people use it differently. In a way it's unimportant, not least because it's literally invisible and so often overlooked. Yet it continues to cause problems which repeatedly waste time, and is especially distracting and troublesome for the many detail-oriented developers among us.

Clearly the only sane way to use whitespace in Perl is:

• 4 character indent
• only use spaces, never use tabs
• never allow any trailing whitespace

One of the worst violations of these rules is an indent composed of mixed tabs and spaces. This is a list of reasons why that's very wrong:

• When someone gets fed up and changes that whitespace to make it conform, it introduces noise into the code diff (they don't create a separate branch due to the overhead costs in a bureaucratic, audited environment).
• Different text editors and web interfaces all display the indent differently, making the code hard to follow.

So, you're thinking of switching to git...

I've written a few things I learnt about using git:

• Git workflow best practice
• Git hooks, config and prompt

I've also gathered a few links over the years, many relating to the philosophy and strategy of git. Some highlights are:

• A successful Git branching model
• A pretty intense dynamic git cheatsheet
• Git from the bottom up (PDF)
• The git parable
• Complete "Pro Git" book

The git features I like best are:

• git stash
• Quickly store a copy of all your unsaved changes, so you can work on them later
• You can use it like: git stash save "halfway through debugging issue foo"
• git branch
• Unlike other version control systems, git makes it incredibly easy, fast and fun to create a new branch for some changes you've made
• Everything is stored locally until you decide to "push" it to the main repo
• To get the most out of git, frequently commit all your work to some branch or other (at least once a day)
• git reflog
• See a history of (almost) all the git commands you issued
• In case you badly mess up you can save your life here
• git rebase -i
• Interactively squash together several commits
• Change commit messages
• Even split out a large commit into several different commits
• git log --color --stat=200,200 --decorate --abbrev-commit --relative
• Improved log

I publish my git config.

Git cache password over https

error: cannot run git-credential-cache--daemon: No such file or directory
fatal: unable to start cache daemon: No such file or directory

# fix:
wget http://example.com/git-daemon-1.7.12.4-0.7.1.x86_64.rpm
sudo rpm -i git-daemon-1.7.12.4-0.7.1.x86_64.rpm
git config --global credential.helper cache
git config --global credential.helper 'cache --timeout=28800' # 8 hours

(source)

Oracle clients

A short list:

• dbVisualizer
• Toad
• Oracle SQL Developer

Jenkins admin notes

Some things to help on your Continuous Integration journey with Jenkins:

• Use  docker pull jenkins  for a quick start
• Security can be a bit fiddly to set up. Try these easy settings:
• Jenkins’ own user database
• Allow users to sign up
• Logged-in users can do anything
• There is a Role-Based Strategy plugin for more advanced use
• If you accidentally lock yourself out, edit  $JENKINS_HOME/config.xml  to say false and restart Jenkins
• To see a list of failing tests, configure a Post-build action for  "Publish JUnit test result report"

-- Jenkins ver. 1.642.2

Simplest easiest quickest web server script

cd ~
python -m SimpleHTTPServer 8080

(source)

PHP, pecl, phpize PDO, mysqli for Perl devs

"pecl" is the same as "cpan"
"phpize" is the equivalent of "perl Makefile.pl"
"PDO" is PHP's DBI equivalent (i.e. general purpose RDBMS connector)
"mysqli" is the more modern version of the mysql-specific connector

Simple perl webserver

HTTP::Server::Simple

(contrast with python one-liner)

#!/usr/bin/perl

# https://metacpan.org/pod/HTTP::Server::Simple
# /versions/perl-5.8.7/bin/perl -MHTTP::Server::Simple test2.pl

use strict;
use warnings;

{
package MyWebServer;

use HTTP::Server::Simple::CGI;
use base qw(HTTP::Server::Simple::CGI);

my %dispatch = (
    '/hello' => \&resp_hello,
    '/' => \&resp_index,
);

sub handle_request {
    my $self = shift;
    my $cgi  = shift;

    my $path = $cgi->path_info();
    my $handler = $dispatch{$path};

    if (ref($handler) eq "CODE") {
        print "HTTP/1.0 200 OK\r\n";
        $handler->($cgi);

    } else {
        print "HTTP/1.0 404 Not found\r\n";
        print $cgi->header,
              $cgi->start_html('Not found'),
              $cgi->h1('Not found'),
              $cgi->end_html;
    }
}

sub resp_hello {
    my $cgi  = shift;   # CGI.pm object
    return if !ref $cgi;

    my $who = $cgi->param('name');

    print $cgi->header,
          $cgi->start_html("Hello"),
          $cgi->h1("Hello $who!"),
          $cgi->end_html;
}

sub resp_index {
    my $cgi = shift;
    print $cgi->header;
    open(my $fh,'<','index.html') or die "can't open index.html";
    local $/ = undef;
    my $out = <$fh>;
    close($fh);
    print $out;
}

}

# start the server on port 8080
my $pid = MyWebServer->new(8085)->background();
print "Use 'kill $pid' to stop server.\n";

(source)

Change quoting level in Outlook 2007

When replying to an HTML email, the original message has a blue bar to the left (which replaces the traditional > symbol). I've seen in previous versions of Outlook that pressing enter on this line takes you to a new line without the blue line ("paragraph unindent") and allows you to write a reply inline. That's fine and what I'd expect.

But currently that doesn't work and I can't easily get rid of the blue line, meaning all my inline replies look like they're part of the original message! Quite odd. Here's how to work around the problem:

• Change the message format from HTML to Rich-text
• Type Ctrl-Q where you want to add a reply
• Type your reply

Voila

(source)