some jmeter conveniences

In jmeter I recently had reason to set these (can be in jmeter.properties or user.properties).

 

CookieManager.save.cookies=true
view.results.tree.max_results=0

 

The first is to have cookies be available as variables and the second is because (to save RAM, probably) jmeter GUI doesn't show everything when you load a jtl file. The second makes it show all results.

Apparmor and empty man pages on old kernels

 On my odroid HC4, the kernel is the stock 4.9.x from hardkernel.  I haven't gotten around to switching to mainline since I do like petitboot and don't want to have the little box connected to a monitor while figuring things out.  It does work as-is, just with some small issues.

For instance, man doesn't render man pages. 

https://bugs.launchpad.net/ubuntu/+source/man-db/+bug/1778684

The workaround is to disable apparmor for /usr/bin/man.  I should just upgrade the kernel, really.  But for now:

apt install apparmor-utils
aa-disable /usr/bin/man

odroid hc4 and zfs

 The HC4 odroid images (official focal and meveric's armbian-buster (and I think toogood's images too) don't have zfs in the kernel (I think because zfs isn't in the mainline kernel either?)

Installing zfs-dkms fails because there's some incompatibility between zfs-dkms 0.7.12 in focal and buster.

I had a play with git clone and building openzfs master.  That correctly built the kernel module and importing the pool and running zpool and zfs-test all succeeded.

Meveric, however, has pointed me at:

apt install -t buster-backports zfs-dkms

Because

"I checked however on the ZFS DKMS module form Debian Buster and found the version that is in the default repository (version 0.7.12 of spl-dkms and zfs-dkms) are not compatible with Kernel 4.9 that we're using.
Luckily for you in buster-backports there's version 0.8.5 available and THAT is compatible with Kernel 4.9."

 Many thanks to Meveric for the assist.  While I could have continued with what I'd built from source, it's better to use the backported spl-dkms so that I can get future updates/fixes.

https://forum.odroid.com/viewtopic.php?p=313882#p313882

Setting up devstack on my bionic-base Virtualbox

[WIP]

 These are just personal notes, probably not relevant to *you*.

I have an Ubuntu Bionic base VirtualBox instance that I clone to build other bionic instances.

Right now, I'm trying to get devstack running in a clone of that base.  That failed with an error around not being able to do something with python simplejson.  My current attempt to fix this is to purge the python3-simplejson package, unstack.sh and then run stack.sh again.

Removing hacking attempts from web logs

I needed to extract valid http request URLs from a client's web access logs (IIS).  It wasn't sufficient to whitelist known good URLs since there were many URLs that attempted directory traversal or executing php scripts from legal looking URLs. 

A simple solution came to mind after some percolating.  I'd extracted timestamps, paths, HTTP method, remote IP address (client) and a few other things.  I didn't need to have perfect logs (didn't need to correctly identify every single log entry).  Having a few false positives and a few false negatives was fine.

Identifying IP addresses used by invalid requests (in my data set, anything with .. or php in them were invalid, the first being a sign of someone attempting directory traversal attacks and the second being a sign of someone probing for vulnerabilities in many php applications.

delete from [table] where ip in (select ip from [table] where url like '%..%' or url like '%php%')
worked very well.

Cisco AnyConnect cert errors on Ubuntu

I needed to use Cisco AnyConnect to connect to a client's VPN server.  Whenever I'd start up the AnyConnect GUI though (server hostname already entered in the server textbox), it would complain about the server cert being untrusted.

There are a few online posts about symlinking things from firefox into the correct system lib directories for the correct linux arch. 

I didn't want to do that (because I wasn't clear what else would break), so I ran strace -f on the vpn GUI and looked for interesting open* calls.  It turns out there are open calls to, e.g., ""/root/.mozilla/firefox/".  That happens before the open calls on /opt/.cisco/certificates/ca*.

On this box (an lxc instance just for this vpn) I'd never run firefox as root so there was no .mozilla/firefox directory at all.  I started up firefox as root but didn't browse anywhere and after that the VPN no longer warns about the cert being untrusted.

The Cisco anyconnect software is installed from a self-uncompressing shell script, so it's not going to know where the system it's installed on stores its certs, so I guess it just punts and looks in root's firefox (plus a few other places that aren't set correctly for it on Ubuntu) for certs.

Insightful forum post on Shogun 2 sieges

https://gaming.stackexchange.com/questions/80775/tactics-on-how-to-siege-a-castle-in-shogun-2

Firefox and Jmeter TLS 1.4

I was using Jmeter 5.1.1 and the latest firefox (linux 69.02 at this writing) and the jmeter generated root cert was breaking firefox.  Firefox was failing with

An error occurred during a connection to [domain].com. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG

This turned out to be answered by:


DanL on stackoverflow


Jmeter is using TLS 1.4 and this firefox version doesn't like that.  The fix, (copied here although I doubt SO will actually go away, props to DanL for the perfect answer):

• Go to about:config in Firefox
•  Set security.tls.version.fallback-limit and security.tls.version.max to 3.

I had a separate issue and I may as well record that here.  Before I found this solution I had a separate issue.   I'd remove the ApacheJMeterTemporaryRootCA.crt and would expect it to be regenerated when starting the jmeter https test script recorder.  I'd get the dialog box indicating it had been generated, but the file wouldn't actually be on the disk.

This is answered by:

Gizit on Stack Overflow

If I want to force actual creation of the crt file I also need to remove proxyserver.jks.  Once that's been removed, starting the https test script recorder generates both proxyserver.jks and the jmeter temporary root crt.

nm-applet on a headless computer

I have a bunch of tiny computers from odroid that run Ubuntu (lately, Bionic).  I don't usually connect them to a monitor at all, they're just servers and their GPUs are wasted.  I wish I could buy these things without GPUs for a little less money :-).

The problem at hand though is that (at least with Bionic, possibly with Xenial too), if I start a vnc server on the box and connect to it, nm-applet won't run in vnc, failing with "Not authorized to control networking".

Fortunately there is a workaround (and I agree with the OP, it's too bad this doesn't ship with at least templates to copy, or disabled configuration in place).

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=643028#59

create (or edit, if already there) /etc/polkit-1/localauthority/50-local.d# cat org.freedesktop.NetworkManager.pkla to contain:

[nm-applet]
Identity=unix-group:netdev
Action=org.freedesktop.NetworkManager.*
ResultAny=yes
ResultInactive=no
ResultActive=yes

git on windows -- file permissions

I had some problems with git on windows reporting some files modified.

git show on those files showed only that file permission modes had changed.  This would likely have been due to some permissions on linux which don't exist on windows.

This is fixed by following denisw's advice

# For the current repository
git config core.filemode false   

# Globally
git config --global core.filemode false

Disabling auto-window maximizing

I use Mate on Ubuntu and I don't run compiz.  This fix is specific to Mate without compiz.

Sometime in the recent previous Ubuntu versions, Unity (and other things that depend on the gnome libs, I guess) implemented window auto-maximizing when their upper edge touches the top of the screen (or the bottom of the top panel).  This may also affect the left and right edges of the screen, but it was always only a problem for me with the top edge.

To fix this in Mate:

System >  Preferences > Look and Feel > Mate Tweak

select Windows

and uncheck enable window snapping.

Reminder link -- bcache suspend resume script

the launchpad bug with the bcache.sh script (and location). Except no, that script isn't called when I pm-suspend.

OOM during rsync with USB drives

I had a (maybe/probably) dying external USB drive. It's part of a non-RAID LVM so I needed to grab everything off it before it died. I mounted a second external USB drive and started an rsync job to copy everything from the failing drive to the new one. After copying around 250GB the rsync would fail with an oom. 

Subsequent attempts would fail the same way but much more quickly. Running rsync with ionice -c 3 doesn't help (suggested in an ubuntuforums post), I wasn't using any of the options that would make recursiveness non-incremental. 

 I tried quite a few things (including --delete-during so that successive rsync attempts would have fewer files to keep track of) but finally found the solution. 

While rsync is running in one screen window, in a separate screen window I run: 

 
while true
do
  echo 3 > /proc/sys/vm/drop_caches
  sleep 2
done
 

There may be interaction between the USB drivers (perhaps timing/slowness?) and the memory used by the buffer caches. Perhaps when rsync needs memory the kernel isn't able to free it fast enough and the oom killer steps in before the memory becomes available to give to rsync? 

In any case, with this drop_caches loop, rsync is now running happily for hours. I expect it'll actually finish. I don't mind the loss of caching. It's only for very large copies from USB drives and the copies are reading everything sequentially.

There won't be much of a benefit to caching. Perhaps I could use just "1" though instead of "3", so that we'd keep inodes and dentries in the cache. But the limiting factor here is really USB transfer rate and the transfer rate while dropping caches is just the same as the rate when not dropping caches (before oom).

Building a mainline kernel for the odroid

Shuah Khan has a howto on building a 4.x mainline kernel for the odroid XU4.  That was very helpful when I built the mainline kernel for my odroid XU3.  I've also got an odroid U3 that I'll be upgrading soon.

I wrote a little shell script to let me do most things automatically (because I needed to rebuild the kernel a few times, adding features that I needed).

Mainly I needed ecryptfs support, iptables modules and some cgroups options for unprivileged lxc containers.  Also a little modification to set the default cpufreq governor to ondemand instead of performance.  I'll update this post at some point to document which additional options I enabled to get all of those things.

I've seen a few more minor patches elsewhere that I'll want to test out.  In particular I'll look for the Mali GPU driver config fix to set it to either not run at all or to run at its lowest energy level (because I don't use that at all, these odroids run headless).

A list of things that I've needed to enable (I'm adding items as I find missing items and rebuild the kernel iteratively :-):

ecryptfs support

• CONFIG_OVERLAY_FS
• CONFIG_ECRYPT_FS

tun device for openvpn

• CONFIG_TUN

IO accounting stuff for iotop

• CONFIG_TASKSTATS
• CONFIG_TASK_DELAY_ACCT
• CONFIG_TASK_XACCT
• CONFIG_TASK_IO_ACCOUNTING

netfilter/iptables modules

• CONFIG_NETFILTER_ADVANCED
• most things in IP: Netfilter Configuration
• most things in IPv6: Netfilter Configuration
• most things in Core Netfilter Configuration

user namespaces for unprivileged lxc

• CONFIG_USER_NS
• 
  
• I forget if anything else was disabled in namespaces support, I just enabled everything under there.

veth support for lxc

• CONFIG_VETH

#!/bin/bash

# optionally, make menuconfig and set options.
# particularly for the cgroups stuff for unprivileged lxc and ecryptfs

make menuconfig

#make exynos_defconfig

make prepare modules_prepare
time make -j 8 LOCALVERSON="-tiger" bzImage modules dtbs

cp arch/arm/boot/dts/exynos5422-odroidxu3.dtb /media/boot/exynos5422-odroidxu3_next.dtb
cp arch/arm/boot/zImage /media/boot/zImage_next

make modules_install

apt-get install -y live-boot

cp .config /boot/config-`cat include/config/kernel.release`
update-initramfs -c -k `cat include/config/kernel.release`
mkimage -A arm -O linux -T ramdisk -C none -a 0 -e 0 -n uInitrd -d /boot/initrd.img-`cat include/config/kernel.release` /boot/uInitrd-`cat include/config/kernel.release`
cp /boot/uInitrd-`cat include/config/kernel.release` /media/boot/
cat include/config/kernel.release

echo "edit /media/boot/boot.ini"
echo "comment out the current setenv bootcmd and replace it with"

echo "setenv bootcmd \"fatload mmc 0:1 0x40008000 zImage_next; fatload mmc 0:1 0x42000000 uInitrd-`cat include/config/kernel.release`; fatload mmc 0:1 0x44000000 exynos5422-odroidxu3_next.dtb; bootz 0x40008000 0x42000000 0x44000000\""

x11vnc xauth

I've had problems lately starting x11vnc (to connect to :0.0) when I'm starting it over ssh.  I don't want x11vnc running all the time, so I don't start it when I'm at work.  But when I need it I'm *not* on the desktop in question, instead I'm remote.

I found the solution at ubuntuforums:

http://ubuntuforums.org/showthread.php?t=1314958

I do run screen on the remote desktop (my work desktop), and I generally start screen when X is already up, so *that* has the correct $XAUTHORITY entries.  So I can just connect to the running screen instance and do:

echo $XAUTHORITY

That will have something (from the ubuntuforums link above):

/var/run/gdm/auth-for-fred-s0G6r1/database

and, from ssh, I can use that:

x11vnc -display :0 -auth /var/run/gdm/auth-for-fred-s0G6r1/database

I should probably just have .bashrc detect if there's a $XAUTHORITY variable and if there is, echo it out to a file that I can then just source :-).

btrfs COW breaks Virtualbox

btrfs copy on write breaks VirtualBox so that starting a VM will often not complete due to spurious disk errors.

I'm trying out the chattr fix that turns off COW.

I've just done this:

cd ~/VirtualBox\ VMs

for f in `find . -name '*.vdi'`

do

  F=$f

  T=tmpfile

  touch $T

  chattr =C $T

  dd if=$F of=$T bs=1M

  rm $F

  mv $T $F

done

Here's hoping it all works :-).  It's faster than mkfs.ext4 and restore from backup :-).

Lightning floating timezone warning

For the longest time (because I'm lazy), everytime thunderbird+lightning would start it would show three dialog box warnings about Unknown timezones being treated as the "floating" timezone.

This is easily fixed (in the sense that the warnings don't show anymore, and, given the nature of the fix, likely in the sense that timezone handling is now correct).

1. Edit | Preferences | Advanced | Config Editor
2. Click on the "I'll be careful" button.
3. search for calendar.icaljs boolean value
4. Set the value to true.

attic prune when using a single repository for multiple hosts

I use Attic Backup.  I'm considering switching to the BorgBackup fork, but that'll be for the future.  Maybe when BorgBackup gets pull backups implemented.

I have multiple computers (some servers, some desktops/laptops).  All of them backup to the same attic repository.  This is mainly to take advantage of deduplication.  Some files are sufficiently important to me (and disk space is now sufficiently large) that I keep the important files synced on ALL my machines, servers, laptops and desktops alike.

The disk savings are pretty large and pretty obvious a reason for having just one backup repository.  There are disadvantages to this too.  Attic locks the repository when doing any work on it (including just listing the repository or showing information about individual archives).  So I have to schedule backups so that they don't interfere with each other.

Backups of the servers run fast enough (after the first 20+hour full backup) now that I can just set up their schedules 2 hours apart and be confident that the second won't need to wait in the queue for the first.  And if it has to, it's no big deal.

The laptops don't have daily backups (mainly because I haven't figured out yet how to get anacron to start up a backup when the laptop wakes up from suspend AND hasn't done a backup yet for that day).  I'll probably just run those backups once a week.  Or maybe I'll figure out how to get anacron to do it.

I also use

attic prune -v -d 30 -w 5 -m 12 -y 100 [REPOSITORY]

to prune old backups.  What this does is keep 30 daily backups, 5 weekly backups and so on.  The problem with this command line is that if I ran backups for server1, server2, laptop1, and laptop2 on the same day then only one of those backups will be kept.  Attic doesn't know which backups were for which servers.  It just knows which backups were done on which day, so it chooses one backup for the day and prunes/deletes the others.

The solution is to use the -p parameter.  -p [PREFIX] tells attic prune that it should ignore all backups except the ones that start with [PREFIX].  My backup archives are named, e.g.,

server1-2016-02-09

so in the scheduled shell script that runs the backup, before attic create runs, first attic prune runs:

 attic prune -v -d 30 -w 5 -m 12 -y 100 -p server1 [REPOSITORY]

This will consider only archives whose archive tag start with server1.  So archives for other servers and laptops are safe and are kept.  Only archives of the pattern server1* are considered for pruning according to the rules given on the command line.

Java8+Minecraft server prerequisites

I had to set up a minecraft server on a new VPS when the old VPS gave up the goat.

The new VPS is still Trusty (14.04.3 now).

This setup is a bit better than previously since it's a Xen instance, has twice the RAM and I'm now able to run lxc (and, for minecraft, use an unprivileged lxc container). I had a backup of the minecraft server, so just copied that over. I then installed Oracle's Java 8 since that's what minecraft recommend.

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install libxrender1 libxtst6 libxi6

The last line above is needed since there's no headless option for java8 and when it starts it just requires some shared objects from those packages so it can link against them even though it'll never use them (since this is a VPS, and so, headless).

minecraft is running again now.  We'll see whether the 2Ghz opterons on there are fast enough to keep up with the boys' redstone creations.

I am going to *HAVE* to set up an hourly backup regime though, with hourly rsync.  The price of going with the cheapest VPS providers out there, they might not survive more than 6 months or so, and I need to be able to restore things on the next VPS in the chain.

Thunderbird corrupted panacea.dat causing slowness

I had major thunderbird slowness issues. Just scrolling the folder list (grabbing the folder list scrollbar and moving it, or moving the scrollwheel while in the list) would lead to no thunderbird screen updates for a second or two. Clicking on a mailbox did the same, clicking on messages took even longer. Some discussion on https://bugzilla.mozilla.org/show_bug.cgi?id=794401 led to me removing panacea.dat. This could have been the wrong thing to do, alta88 does say that folder pretty names might be lost. I'm on thunderbird 38.4.0 though, not the 12,13,14,15 versions discussed in the bug report. That may be why, after just removing the file, I didn't lose any pretty folder names. That certainly has sped up thunderbird so it's now usable and now I won't need to switch to evolution :-)