A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

Writing a Jenkins plugin for dummies


This guide is written for Ubuntu

Install Jenkins
  • Follow these steps
  • It will install this file amongst others: /usr/share/jenkins/jenkins.war
  • You may need to start Jenkins on a different port than its default 8080
  • The config is in this file: /etc/default/jenkins
    • There are a list of command-line options at the end of this file
  • You run it with this command:
    • java -jar /usr/share/jenkins/jenkins.war --httpPort=9090
    • (however this doesn't use the config you just edited)
Write a plugin



Allow PRE text to wrap on Confluence wiki


Add this to the wiki code:


{html}
<style type="text/css">
/* Allow PRE elements to wrap --- better than manually adding ad-hoc
 * linebreaks because
 * (a) It's automatic
 * (b) It adapts to the window width
 * (c) People can still copy-and-paste from the browser window and get the
 * original long lines
 */
pre {
 white-space: pre-wrap; /* css-3 */
 white-space: -moz-pre-wrap; /* Mozilla, since 1999 */
 white-space: -pre-wrap; /* Opera 4-6 */
 white-space: -o-pre-wrap; /* Opera 7 */
 word-wrap: break-word; /* Internet Explorer 5.5+ */
}
</style>
{html}

Thanks duckbill


Your PC does not need dynamic wallpaper or a health check

Do not install any of the following types of programs, they are almost guaranteed to be spyware, slow down your computer, and be difficult to remove:
  • Dynamic wallpaper/screensaver
  • PC health check
  • Speed up your PC
  • Clean your PC
  • Any new special anti-virus program (you should only have one trusted anti-virus program)
  • Defrag your computer
Instead, use Ad-aware or Spybot Search & Destroy to keep your Windows computer clean.

Windows "add or remove programs" list is empty

First of all, just wait for a few minutes. The list may appear after some time.

If still no luck:
  • Highlight and copy the entire bolded text below: 
    • cmd /c REGEDIT /E "%userprofile%\desktop\uninstall.txt "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\ 
  • Click on Start > Run and paste in the text you have just copied. Hit Enter. 
  • A notepad document should appear on your desktop called uninstall or uninstall.txt 
  • Open it to view the contents
    • Look for the "UninstallString" values and copy those into the Start > Run box to uninstall programs


Thanks to tomshardware.co.uk

How to manage problematic application windows in Ubuntu

In Ubuntu:

To find out what application a window belongs to: Type xwininfo in the console, then click on the window.

To kill a window, type xkill in the console, then click on a window to kill it.


Complex joins with DBIC in Perl

In a DBIx::Class ResultSet, sometimes you want to return all rows from aaa that have foreign keys to rows in bbb, that have foreign keys to rows in ccc.

This returns rows from ccc which is not what you want:

    return $_->search({
        some_id => { '!=' => undef },
    })
        ->search_related('aaa')
        ->search_related('bbb')
        ->search_related('ccc');


...so rather than doing it in a Perly way which returns an array not a resultset, not to mention many more queries than strictly necessary:

    return grep {
        $_->search_related('aaa')
            ->search_related('bbb')
            ->search_related('ccc')->all;
        } $self->search({
             some_id => { '!=' => undef },
        });


...instead do it the DBIC way, which returns a resultset, and is further chainable, etc.:

    return $self->search({
        'me.some_id' => { '!=' => undef },
        'ccc.some_other_id' => { '!=' => undef },
    }, {
        join => { 'aaa' => { 'bbb' => 'ccc' } }
    });


See docs: DBIx/Class/Manual/Joining.pod#COMPLEX_JOINS_AND_STUFF

Ubuntu: Remove Unity and overlay scrollbars

1. Select 'Ubuntu classic' instead of 'Ubuntu' at the login screen

2. sudo apt-get remove overlay-scrollbar

Always get a stack trace when Perl dies

Always get a stack trace when Perl dies or warns:

use Carp::Always;
or

perl -MCarp::Always -e'die "argh"'


Easily switch communication devices in Windows 7

http://coastalrocket.blogspot.co.uk/2009/12/switching-audio-devices-in-windows-7.html

Get unique values from an array in Perl

# Unique an array

@unique_list = keys %{{ map { $_ => 1 } @big_list }}; # aka dedupe a list

# Unique a hash on a specified key 'id'

@unique_list = values %{{ map { $_->{id} => $_ } %big_hash }};




Explanation

Extract an RPM file

How to see what's in an RPM file:


rpm2cpio /path/to/file.rpm | cpio -i --make-directories


Thanks to askubuntu.com

View current queries in postgres

Edit config

sudo vim /var/lib/pgsql/9.0/data/postgresql.conf


Add this line:

stats_command_string = true


Restart

pg_ctl reload


Then

SELECT datname,procpid,current_query FROM pg_stat_activity


Thanks

Sublime text editor tricks

Sublime is not a bad text editor for Linux:

Mount a remote directory:  sshfs username@hostname:/remote/path /local/path

Put a vertical line after 80 characters: Preferences > Settings - User > ... add an entry: "rulers": [80]

Change syntax highlighting coloursPreferences > Browse Packages > Color Scheme - Default > ... edit the Color Scheme file you are using.

{
"color_scheme": "Packages/Color Scheme - Default/All Hallow's Eve.tmTheme",
"font_size": 10,
    "rulers": [80],
    "auto_complete": false
}


Shortcuts:

  • Shift-F11 = Enter "concentration" mode AKA "distraction free" mode

wget through a proxy

Edit /etc/wgetrc or ~/.wgetrc

Add these lines:

https_proxy = http://username:password@host:port/
http_proxy = http://username:password@host:port/
ftp_proxy = http://username:password@host:port/

Vim experiment: Disable arrow keys


" EXPERIMENTAL

" Disable the arrow keys so you're forced to use h,j,k,l
" and other key bindings and will therefore be more productive
" an experiment

"  inoremap  <Up>     <NOP>
"  inoremap  <Down>   <NOP>
"  inoremap  <Left>   <NOP>
"  inoremap  <Right>  <NOP>
"  noremap   <Up>     <NOP>
"  noremap   <Down>   <NOP>
"  noremap   <Left>   <NOP>
"  noremap   <Right>  <NOP>

" cf the default statusline: %<%f\ %h%m%r%=%-14.(%l,%c%V%)\ %P
" format markers:
"   %< truncation point
"   %n buffer number
"   %f relative path to file
"   %m modified flag [+] (modified), [-] (unmodifiable) or nothing
"   %r readonly flag [RO]
"   %y filetype [ruby]
"   %= split point for left and right justification
"   %-35. width specification
"   %l current line number
"   %L number of lines in buffer
"   %c current column number
"   %V current virtual column number (-n), if different from %c
"   %P percentage through buffer
"   %) end of width specification
"set statusline=%<\ %n:%f\ %m%r%y%=%-35.(line:\ %l\ of\ %L,\ col:\ %c%V\ (%P)%)

Virtual machines on linux

VirtualBox OSE
  • Free
  • Works
  • Have to shut VM down to change any settings
  • Command line access to read settings:
    • VBoxManage list vms
    • VBoxManage showvminfo [vm name]
  • Can use NAT and port forwarding to access locally when on wifi
  • Embiggening a disk is tedious
VMWare
  • Very big and heavy
  • Client is free
  • Software to create VMs is paid

irc bots

Infobot!

# karma
[name]++ # [reason]
[name]-- # [reason]
explain [name]

# concepts
[concept]?
[concept] is [explanation]
no, [concept] is [explanation]
[concept] is [suggestion]
[concept] is also [alternative explanation]

Perl: Subtract one list from another list

my @difference = grep { not ($_ ~~ \@small_array) } @big_array;


# ~~ is the smart matching operator

Easily create graphs from code

Use graphviz:

1) write graph code

digraph g{
  Opened[label="1\nOpened\nE: open door"];
  Closed[label="2\nClosed\nE: closed door"];
  node[shape=plaintext];

  Opened -> close_door[arrowhead=none];
  close_door -> Closed;
  Opened -> open_door[dir=back];
  open_door -> Closed[arrowhead=none];
}


2) generate graph

dot -Tjpg -ograph.jpg graph.dot

3) view graph


You can easily store the source code to the graph in version control.
Don't forget to also store the instructions to rebuild it.

See SQL commands that DBIx::Class is generating

Set environment variable:
    DBIC_TRACE=1

Pretty printing is available, as of DBIx::Class 0.08124:
    DBIC_TRACE_PROFILE=console DBIC_TRACE=1

(sources: CPAN docs, and a foolish manifesto blog)

For plain DBI calls, see also:
    DBI_TRACE=1

(source)

Debugging Test::Class tests in Perl

Only test one method:
TEST_METHOD="Class::To::Test" prove run_all_tests.t

Script to test one class only:

#!/bin/bash


MODULE=$1
FILE="/tmp/runtest"
rm -f $FILE
echo '#!/usr/bin/perl' >> $FILE
echo "use Test::Class;" >> $FILE
echo "use Test::NoWarnings;" >> $FILE
echo "use $MODULE;" >> $FILE
echo 'Test::Class->runtests;' >> $FILE


prove -lv -It/lib -It/unit-tests/classes $FILE

Run like this:
script.sh Class::To::Test

You can combine the two commands to test one method from one class only.

Enable/disable Lenovo touchpad

Press Fn + F8

(F8 should have a picture of a touchpad with a line through it)

Perl debugger tips and tricks


In the debugger:

  • $DB::deep = 500 # prevent the warning "100 levels deep in subroutine calls!"
  • {{v # display the code ahead after every step
  • b Test::Something::App::Module::login # break at a particular subroutine in a specified module
  • c # continue until the next breakpoint
Also remember that the debugger often provides extra stack trace information for 'untraceable' warnings or errors.


Ruby basics


Save URL to a file

require 'open-uri'
open('image.png', 'wb') do |file|
    file << open('http://example.com/image.png').read
end

is this the same as:

require 'open-uri'
file = open('image.png', 'wb')
file << open('http://example.com/image.png').read

Fetch URL through a proxy

print Net::HTTP::Proxy(proxy_addr, proxy_port, proxy_user, proxy_pass).get URI.parse('http://www.compufer.com/')

Hello world

puts "Hello, what's your name?"
STDOUT.flush
name = gets.chomp
puts 'Hello, ' + name + '.'

Blocks


def try  
  if block_given?  
    yield  
  else  
    puts "no block"  
  end  
end  
try # => "no block"  
try { puts "hello" } # => "hello"  
try do puts "hello" end # => "hello" 

Notes

  • Whichever variables(s) you pass into the block, the method will fill them with something, depending on what its function is. e.g. "open" puts a file handle in there.
  • List of file modes
  • HashHashes
  • ---------------
  • Builder tut1tut2tut3

Make apt-get and aptitude work through a proxy

1) sudo vi /etc/apt/apt.conf.d/70debconf
2) It should already have: DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt || true";};
3) Add this after what's there (no blank lines):
Acquire {
        http {
                Proxy "http://user:pass@host:port";
                No-Cache "false";
                Max-Age "86400";
                No-Store "false";
        };
};
Thanks

Handle the database with Test::Class

package My::Test;

use base 'Test::Class';

sub startup : Tests(startup => 1) {
    $schema = NAP::PRL::Schema->connect( $dsn, $user, $password );
}

sub setup : Test(setup) {
    $schema->txn_begin;
}

sub teardown : Test(teardown) {
    $schema->txn_rollback;
}

Which version of Ubuntu?

Which version of Ubuntu should I install?

  • 11.04 - Yes! This works fine.
  • 11.10 - No. The menus are borked.

These are all the lovely settings you're missing in >11.04:

System preferences:



System administration:



Control centre:

Enable scrolling in GNU screen

.screenrc: 
termcapinfo xterm* ti@:te@ 


Install ruby gems

sudo yum -y install gcc ruby-devel rubygems
sudo gem install nokogiri

DBIx::Class basics

# SELECT COUNT(*) FROM product WHERE id = 104316
my $rs = $schema->resultset( 'Public::Product' )->search( { id => 104316 });
print "test: ".$rs->all;

Perl Webform CRUD

I have a dream that I can use Perl to build a webpage that allows me to view what's in a database table, and to add new rows to it, *without specifying any of the fields or constraints*!

Okay, maybe I might want to map column names to user-friendly labels, or add additional constraints to the fields (e.g. email address), but that's it.

I particularly don't want to duplicate the constraints which I've already programmed into the database table once. i.e. The form should only validate input data against the database schema, and feed back appropriate errors/warnings.
I don't want to have to duplicate the column list in the database, by defining all the form fields in a configuration file.
And I definitely don't want to have to write any HTML at all.
I must be able to completely integrate the system into my own framework.

UPDATE:
Perhaps if I don't specify pretty names for the columns, or their order, then I end up with a database management tool like phpMyAdmin. This means configuration of the fields is necessary for a real web app.
Probably the same goes for HTML output - its purpose is to lay out the page, so I am required to list the fields there as well.
But I really dislike having to edit all three files (schema, config, and template) when I add a new database column.

Possible modules for DBIx:
  • Catalyst::Manual::Tutorial::09_AdvancedCRUD::09_FormFu - Handles the database part automagically. We define the webform fields, along with any additional constraints and helpful validation features in a configuration file. The fields must also be defined in the template. However, error messages defined in the config file will appear automatically in the right place in the form (with a little help from CSS).
    • HTML::FormFu - This seems to be the best thing.
    • Catalyst::Model::HTML::FormFu (deprecated)
    • Catalyst::Model::FormFu - "Speedier interface to HTML::FormFu for Catalyst"
  • HTML::FormFu::Model::DBIC - "Integrate HTML::FormFu with DBIx::Class". The form fields must be specified in config, and they must match the DBIx class schema.
  • Catalyst::Plugin::FormBuilder - "merges the functionality of CGI::FormBuilder with Catalyst and Template Toolkit" - ?
    • Catalyst::Controller::FormBuilder
  • Catalyst::Example::InstantCRUD - contains DBIx::Class::InstantCRUD. Presumably this is meant to be a database admin tool. See instantcrud.pl
  • Catalyst::Plugin::CRUD::DBIC - "CRUD plugin for DBIx::Class". Looks promising, but the fields also must be listed in an HTML template.
  • DBIx::HTMLinterface
  • DBIx::HTMLView 
  • DBIx::Class::HTMLWidget
  • DBIx::Class::FormTools
  • Catalyst::Helper::Controller::Scaffold
  • Catalyst::Enzyme 
For CGI:
  • CGI::Application::Plugin::BREAD 
  • CGI::Prototype 
  • CGI::FormBuilder - "I hate generating and processing forms" (the author). But do we still have to specify the form fields using Perl code?
See also:
Modules I've actually seen in use:

  • HTML::FormHandler - example
  • Data::FormValidator

Git basics

Have git remember merge conflict resolutions so you never have to do them more than once:
git config --global rerere.enabled 1

Prevent "git push" with no parameters from pushing anything other than your current branch:
git config --global push.default current

Show a graphical summary of the merge tree (looks a bit like gitk).
This gives the answer to "Have I pushed or not"? :
git log --decorate --color --graph  --oneline


Find the point at which you forked (branched):
git merge-base [branch] [trunk or previous branch]


Show all changes made in this branch (if you branched from master):
git log --oneline --name-only --reverse master..HEAD


Ensure only one commit per file; pipe the above through:
| grep -v '^....... ' |sort |uniq -c |sort -nr 
 
Edit all commit messages since a particular commit:
git rebase -i $(git merge-base [branch or HEAD] [trunk or previous branch])


Undo the last commit made locally (e.g. in order to re-do it differently, perhaps in the middle of a rebase): 
git reset HEAD^


To reverse a pushed commit two commits back,
first reverse the second most recent commit, and then commit it back:
git revert -n HEAD~1
git push origin branch
(Do not try to rebase what has been pushed, it will not work. Rebase only changes your local repo)

To revert a merge commit:
git revert SHA -m 1
(Where SHA is the commit ID)

Reset a branch to be same as the repo:
git reset --hard origin/branch_name

Delete a branch:

git branch -D branch_name # delete it locally
git push origin :branch_name # delete it on remote


Use a branch in a different repo for the same project:
git remote add repo_shortname user@server:/repo_name
Use different branch names in each repo.
Push to your repo_shortname instead of origin.

Create a new, tracked branch:
Tracked means that 'git status' will show you how many commits behind the parent branch you are, and if the parent branch has diverged (which means you might want to rebase onto it)
git checkout master # or wherever you want to branch from 
git checkout -t -b my_new_branch

Easily make an alteration to a previous commit:
git commit -m'fixup! same title as another commit' filename
git rebase -i --autosquash [commit id]

(the commit will be put in the correct place and marked as 'fixup' automatically)

A git workflow:
(dca-73 is the name of a branch)
git fetch
git checkout dca-73
git pull origin dca-73
# run tests
# make changes
# run tests
git fetch
git push origin dca-73
git checkout master
git pull origin master
git merge --no-ff dca-73

# run tests

# check it looks right
git log --decorate --graph --oneline

git push origin master




How to merge conflicts without editing the file:

git checkout --ours path/to/filename
git checkout --theirs path/to/filename


Automatically run checks on your code before committing with git hooks:
e.g. Just edit the file .git/hooks/pre-commit in your working directory, and make it executable.
It will not be committed along with your code because it's in the special .git directory.

PostgreSQL basics (for MySQL users)

Installing

  • sudo apt update
  • sudo apt install postgresql postgresql-contrib
  • sudo service postgresql start
  • sudo passwd postgres # then close and re-open the terminal(?)
  • sudo -u postgres psql
    • create user foo;
    • alter user foo with superuser;
    • alter user foo with password 'new_password';
  • psql -Ufoo -d postgres

Reconfigure authentication if necessary to either require or disable passwords.


Using


MySQL and Postgres command equivalents (mysql vs psql)

connect: psql -U [username, e.g. postgres] -d [database]
  • \c dbname = connect to dbname
  • \l = list databases:
  • \dt = describe tables, views and sequences
  • \dt+ = describe tables with comments and sizes 
  • \dT = describe Types
  • \di = describe indexes
  • \q = quit
  • \x = toggle equivalent of adding MySQL's \G at the end of queries to display columns as rows 
  • \connect database = change to a different database
  • CREATE DATABASE yourdbname;
  • CREATE USER youruser WITH ENCRYPTED PASSWORD 'yourpass';
  • GRANT ALL PRIVILEGES ON DATABASE yourdbname TO youruser;
  • Use auto_increment in a column definition:
    • create sequence foo__id__seq increment by 1 no maxvalue no minvalue start with 1 cache 1; 
    • create table foo ( id integer primary key default nextval('foo__id__seq') );
  • Reset an auto_increment counter (sequence):
    • SELECT setval('sequence_name', 1, false); -- this works
    • ALTER SEQUENCE sequence_name RESTART WITH 1; -- this also works
  • See what's in an ENUM
    • SELECT enumlabel  FROM pg_enum WHERE enumtypid = 'myenum'::regtype ORDER BY id;
  • Closest equivalent of MySQL's "show create table"
    • pg_dump -U postgres --schema-only [database] >> dump.sql
  • See current activity
    • SELECT * FROM pg_stat_activity (equivalent to MySQL's "show processlist")
  • Drop all current connections
    • SELECT pg_terminate_backend(procpid) FROM pg_stat_activity WHERE datname='foo'; # foo = database name
Other stuff
  • pg_dump -Upostgres -hHOSTNAME DBNAME -fOUTPUTFILE.sql --no-password
  • psql -Upostgres -hHOSTNAME -dDBNAME -fINPUTFILE.sql
  • psql -Upostgres -hHOSTNAME -dDBNAME -c "Some SQL command"
      See also

      Audio in Ubuntu

      Don't do this:
      sudo apt-get remove pulseaudio gstreamer0.10-pulseaudio
      
      
      Something:
      $ sudo dpkg-reconfigure linux-sound-base

      Volume control:
      $ alsamixer

      Turn off gmail notification sound in Android

      gmail app version 2.3.5.2:

      • go to inbox
      • press menu | more | settings
      • select your account
      • scroll down to the bottom
      • select 'Labels to notify'
      • here you can change the notification sound or disable it, set vibration and other settings

      CPAN config

      o conf prerequisites_policy 'follow'
      o conf make_install_make_command 'sudo make'
      o conf build_requires_install_policy yes
      o conf commit