Category Archives: Tech

Chrome is Fast, Firefox is Memory Efficient

Chrome is a little faster but uses a lot more RAM. Firefox is a little slower but uses a lot less RAM.

If you browse with only one or a small number of pages open at a time, Chrome is best.

If you browse with 10 or more pages open at a time, Firefox is best. It uses so much less memory, in some cases it runs faster than Chrome because Chrome will consume all your RAM and start swapping long before Firefox does.

On a test I did today on my Ubuntu 15.10 system, with Chrome 46 running with about 12 pages open, overall system RAM usage was 10 GB (and the 8 GB system was swapping). When I opened the same 12 pages in Firefox 42 and closed all Chrome instances, overall RAM usage dropped to 6.5 GB and the system was no longer swapping, so it was much more responsive overall. That’s a savings of 3.5 GB of RAM with only 12 pages open!

The speed difference between Chrome & Firefox is not much on a modern fast system. If you have an older system the difference is more noticeable.

Of course you always need both because there are certain web sites that work only on one or the other. But which will be your default? After making Chrome my default for the past few weeks I’ve switched back to Firefox.

Cyanogenmod 12.1 Battery Life – Fixed!

Some users of Cyanogenmod 12.1 have noticed poor battery life. Recently I encountered this, learned what was causing it and how to fix it. Posting this may help others.

My phone is a Galaxy Note 2 running Cyanogenmod 12.1 nightly build dated Nov 11 2015, but this problem has been reported across a variety of different phones. The GN2 has a big 3100 mAH battery and still makes it through the day despite this bug, but just barely. With this bug fixed, the battery lasts more than twice as long.

Symptoms

Short battery life, phone is always warm when I pick it up, Android’s battery usage screen shows a lot of CPU time spent in Android OS, and the battery life history details (when you touch the battery % graph) shows the device is almost always awake, even when the screen is off and it should be asleep.

Root Cause

If you Google this you can find a more detailed explanation; here is a summary.

The root cause is not in Cyanogenmod, but in Google Apps. The service called Google Play Services uses another service called SystemUpdateService. The notifies the device when a system update is available. This is used mainly by device manufacturers, not by ROMs like Cyanogenmod. The bug is in the Google Play service. It takes a wake lock before checking SystemUpdateService, and if it encounters an error during that check it never releases the wake lock. Basically, some Google programmer didn’t use a try..finally block to ensure the lock is always released. This surely violates Google’s coding standards, but stuff happens – sometimes bugs sneak past code review.

Solutions

I know of 3 ways to solve this. The first didn’t work well for me, but the latter two did.

One solution is to use Cyanogenmod’s Privacy Guard feature to deny wakelocks Google Play Services. This used to work well, but not any more. The current versions of Google Play Services crash when denied the wake lock, which throws a system modal dialog saying Google Play Services has stopped. This re-appears every 5 seconds or so, making the device completely unusable! Yet another violation of Google’s coding standards – if your app can’t get a resource it wants, handle it! Don’t crash and throw a system modal dialog for something as trivial as this.

Another solution is to turn on SystemUpdateService, so Google Play Services no longer fails when it tries to use it, and because it doesn’t fail, it follows the “happy” code path which releases the wake lock it acquired. You might ask, why isn’t this service running already? ROMs like Cyanogenmod don’t use it and usually turn it off to save RAM and battery. If your phone is rooted, you can enable it with this command:

su -c ‘pm enable com.google.android.gms/.update.SystemUpdateService’

NOTE: You might try enabling SystemUpdateService with an app like Disable Service, but I could not because this service never shows up in the list. The above command works whether or not the service appears in the list.

You’ll need to do this every time you boot your phone. You can use apps like Tasker to do that automatically.

The third solution is to prevent Google Play Services from connecting to SystemUpdateService. Apparently, if it can’t connect, it exits cleanly without leaking the wake lock. Yet if tries to connect but the service isn’t running, it does leak the wake lock. In this solution, you can leave SystemUpdateService disabled, which may save a little RAM and battery. The way I do this is with an app called Disable Service. Run the app, select the System tab, select Google Play services, scroll down the list of services it uses (there are hundreds!) to SystemUpdateService, and uncheck it – it will turn red.

You’ll need to do this every time you boot your phone. I haven’t yet found a way to automate it.

Final Note: Google, fix this @#!$%* bug in the Google Play Service! It’s been known for at least a year!!!

Power Moves the Airplane

Actually it’s thrust that moves the airplane, but power determines how much thrust you can get any any given airspeed.

Thrust is what pushes (or pulls) an airplane. Thrust is related to power by a simple equation. Removing constant factors, Power = Thrust * Speed. An engine with a constant power output – like any piston engine – gives max thrust (thus max acceleration) at low speeds, with less and less thrust (thus diminishing acceleration) as speed increases, until you reach top speed, where thrust = drag and acceleration is zero.

Put differently: if someone says, “My engine produces 250 lbs. of thrust” you really know nothing. Why? No engine produces the same thrust at all speeds, and he hasn’t told you at what speed it produces that thrust. But if he says, “My engine produces 250 horsepower” you know everything you need to know. From this you can compute how much thrust it produces at any speed, so given the airplane’s mass and drag coefficient, you can compute its acceleration, climb rate and top speed. In this sense, power is what moves the airplane – it determines how much thrust you can get at any speed.

Another way to think about this: forces move objects. But force alone doesn’t tell us much. Any object that actually moves in the real world, moves a certain distance in a given amount of time. Force * distance is work, and work over time is power.

Example

Consider this from the perspective of the airplane’s propeller. It’s spinning at a certain rate, with a certain amount of torque. Suppose that’s 2,000 RPM and 1,500 ft.lbs. of torque.

That propeller might be powered by an R-1340 piston engine spinning at 2,000 RPM making 1,500 ft.lbs. of torque. Here, the engine is spinning at the same speed as the prop so they can be directly connected, no gearing needed.

Alternatively, that propeller might be powered by a PT6 turbine spinning at 30,000 RPM with 100 ft.lbs. of torque, through a 15:1 reduction gear. The 15:1 reduction gear cuts the RPM by 1/15 and boosts the torque by 15:1, which means 30,000 / 15 = 2,000 RPM and 100 * 15 = 1,500 ft.lbs.

Either way, the propeller can’t tell the difference. Because in fact there is no difference. Either way the propellor spins at 2,000 RPM with 1,500 ft.lbs. of torque, which is about 570 horsepower. The PT6 only has 100 ft. lbs. of torque, which is only 1/15 of the torque of the R-1340. But the PT6 spins 15 times faster, so torque * RPM is the same. In discussing Torque and RPM, neither alone tells you what the engine can do. It is their product, power, that moves the airplane.

Another Example

Suppose someone tells you, “My truck has 500 ft. lbs. of torque”. From this it’s impossible to know how fast it can tow a load up a hill. Towing the load up the hill takes a certain amount of work. You can do that work quickly or slowly. Power is how quickly you do the work. The little electric motor that rolls the truck’s windows up and down can also tow any load up a hill – given enough time, it can do any amount of work. As Archimedes said, “Give me a lever long enough and I’ll move the world”. But if you want to tow a real load up a real hill in the real world, you care how long it takes. Power tells you that.

If that truck makes its 500 ft.lbs. at 3,000 RPM, it has 3 times as much power as it does if it makes the same torque at 1,000 RPM. That means it can tow the load up the hill 3 times faster. Once you know the engine’s power output, you know what it can do. Torque * RPM = Power, and any combination of Torque and RPM that makes that power will do the job.

Back to the Point

In short: power moves the airplane.

More precisely, Power = Thrust * Speed * Efficiency. The reason we must include efficiency will become clear.

First, consider an airplane with a fixed pitch prop. At full throttle standing still it pulls around 2300 RPM – well shy of its 2700 RPM redline. It must be designed this way because otherwise, it wouldn’t be able to fly fast enough. As the airplane starts moving, the prop blades see reduced angle of attack to the oncoming air, which reduces resistance, it would want to spin faster but couldn’t because it would already be at redline. The pilot would have to gradually pull back the throttle during the takeoff roll and climb to avoid over-revving.

Now 2300 RPM is about 85% of the 2700 redline, and since Torque * RPM = Power, assuming flat Torque vs. RPM, the engine is making 85% of its rated power. If you have a 160 HP engine then you have about 135 HP during the takeoff roll, with the engine at full throttle pulling 2300 RPM.

Propeller efficiency is a key factor – it determines how much of the power is converted into thrust. The rest of the power is converted into noise, turbulence and heat. A propeller achieves its ideal efficiency only at a certain angle of attack. This amounts to a medium-ish airspeed. At slow speeds, and at fast speeds, the propeller is less efficient. So our airplane in the above example effectively has less than 135 HP because it’s moving slowly and the prop is gaining efficiency as it speeds up. To be clear, the engine is making 135 HP but some of that power is being converted into turbulence and noise instead of thrust.

Theoretically, since Power = Thrust * Speed, the speed of maximum thrust is zero. But in reality, the speed of maximum thrust is higher. In reality, as you start the takeoff roll, both power and thrust are increasing because the prop is gaining efficiency as you gain speed.

Power (Vy) vs. Thrust (Vx)

Now let’s consider 2 key airspeeds: Vx and Vy. Every pilot knows Vx is the airspeed that gives greatest angle of climb. If you need to clear trees on the takeoff roll, fly at Vx. Vy is the airspeed that gives the greatest rate of climb. If you want to climb to 10,000 feet as quickly as possible, fly at Vy. The difference between Vx and Vy is thrust vs. power. That is, Vx is the speed of maximum excess thrust, and Vy is the speed of maximum excess power. Here, excess means, above the amount needed to sustain level flight. Some facts about Vx and Vy:

  • At sea level, Vx is slower than Vy.
  • As you climb, Vx increases and Vy decreases (as indicated airspeed).
  • Vx and Vy are equal (as indicated airspeed) at the airplane’s ceiling.

One way to think about this: every climb has an angle and a rate of altitude gain. The angle is determined by excess thrust beyond what is needed for level flight. The rate of altitude gain is determined by excess power beyond what is needed for level flight. So, max excess thrust gives the biggest angle and max excess power gives the highest rate.

Recall the drag vs speed curve of an airplane in flight. Induced drag is high at low speeds, low at high speeds. Parasitic drag is the opposite. The speed having least total drag is the point where they are equal. This is usually much slower than you would normally fly. In my 172, it’s about 60 knots. This is also the speed at which you can glide the longest distance: called Vldmax. Now, knowing that Vx is the speed at which you have the most excess thrust, and drag is what saps your thrust, you might expect Vx to equal the speed of minimum drag. It’s close to that, but always slower. Why? Because your engine makes more thrust at lower speeds, and the relationship is linear. As you slow down just a bit from Vldmax, total drag increases less than linearly, while thrust increases linearly. This means thrust increases more than drag, giving you more excess thrust.

Now consider Vy, the speed where you have the most excess power. This is more complex than Vx because it depends on efficiency. First let’s derive the airspeed of minimum power. It is always slower than the airspeed of minimum drag. Let’s start from that speed – Vldmax – and find out why. Suppose it takes power P to fly at Vldmax. How much power would it take to fly just a bit slower? Flying slower, the drag increases a little – but less than linear – so our speed dropped more than our drag increased. Drag = Thrust and Power is Thrust * Speed, and we just saw that speed decreased more than thrust increased, which means their product, power, is smaller. Thus the airspeed requiring minimum power is slower than the airspeed of minimum drag. However, if we continue slowing down, drag will soon increase rapidly requiring more power – not less!

Here is where propeller efficiency enters the picture. If power output is constant, then the speed of maximum excess power is equal to the speed of minimum power required for flight. That would be slower than Vldmax, yet we know Vy is higher than Vldmax! But in reality, power output is not constant because engine power output and propeller efficiency are not constant. Let’s take these separately.

Engine power output: as you speed up from Vldmax, engine power output increases. That is: higher airspeed reduces the angle of attack of the propeller blades. This enables the engine to rev higher at the same throttle setting, which means the same torque at higher RPM, which is more power.

Prop efficiency: at Vldmax, you have poor propeller efficiency. If you speed up, your propeller becomes more efficient.

As you speed up from Vldmax these two factors combine: engine power and propeller efficiency both increase.

Ubuntu 15.10, Thinkpad Carbon X1

WARNING: Right after upgrading I started getting kernel panics 1-3 times per day. This is definitely a problem with the 4.2 linux kernel, as Ubuntu 15.04 with kernel 3.19 (the last version 3 kernel) was rock solid running for weeks at a time under heavy usage, many suspends/wakes, plugging and replugging into docking stations, etc. all without rebooting. I always keep the prior version of the kernel installed, so for me, the solution is to keep Ubuntu 15.10 but select kernel 3.19 in Grub when booting.

Now for the rest of the original post…

I run Xubuntu on most of my computers both at home and at work. Version 15.10 just came out and I’ve upgraded all of them. On a desktop you might not notice much difference. On a laptop it’s got some good stuff you might like.

The biggest difference from 15.04 is the kernel. 15.10 uses kernel 4.2 instead of  kernel 3.19 which 15.04 uses. This new kernel supports the power saving modes of Broadwell processors and micro-hardware appearing on modern laptops.

Another benefit of 15.10, again for laptops, is that tlp is fully supported and in the Xubuntu standard repos. And its default settings work well. It auto-detects when the X1 is plugged vs. running on battery and it extends battery life by a couple of hours – 4-6 hours becomes 6-8 with normal use and screen constantly on.

A bit about tlp… while it comes pre-configured with good default settings, you should check a few things to ensure you’re getting the most from it. Here’s a good tlp reference site to start. Now here’s an important tip from that site, specific to Ubuntu:

First, you must deactivate Ubuntu’s normal CPU controller: sudo update-rc.d -f ondemand remove. This allows TLP to adjust the CPU speeds.
This command reverts this change: sudo update-rc.d ondemand defaults

Use this command to check whether TLP can control the CPU speed: sudo tlp-stat -p. If it says x86_energy_perf_policy: program not installed, then you need to install linux-tools-generic, and ensure the version matches your kernel.

Another thing you should do is allow the CPU speed to vary. The default settings do not. Edit the file /etc/default/tlp as root, and change 4 lines. First, they’re commented out, so un-comment them. And, put in numeric values for your processor.

CPU_SCALING_MIN_FREQ_ON_AC=1000000
CPU_SCALING_MAX_FREQ_ON_AC=3200000
CPU_SCALING_MIN_FREQ_ON_BAT=500000
CPU_SCALING_MAX_FREQ_ON_BAT=2000000

As you can see, I’ve used a lower top speed on BAT, and a  higher bottom speed on AC. This limits performance in favor of battery life, but not too much, while letting it slow down on AC, to save energy or charge faster. However, TLP will only use the higher speeds when the processor is active, so setting higher values won’t necessarily shorten battery life if the machine is idle. That said, TLP allows higher clock speeds liberally even when in powersave mode, so limiting the CPU top speed while on battery does extend battery life.

NOTE: any changes you make to the TLP config file will take effect when you enter the command: sudo tlp start. You can run this command even if it’s already started.

On Ubuntu 15.10 Kernel 4.2 all this happened automatically and everything worked. But when I had to revert back to kernel 3.19, my battery life suffered because TLP was no longer controlling CPU speeds. I had to reinstall linux-tools-generic version like this: sudo apt-get-install linux-tools-generic=3.19.0*, which reverted the version back to match the kernel.

A bit about Xubuntu…

I’ve done 2 in-place upgrades (from 15.04) and 1 fresh install. All went smoothly.

Why Xubuntu? it’s my favorite Linux distro because

  • It’s Ubuntu – the most popular distro with the most complete and up-to-date repositories.
  • It uses the XFCE desktop, which is smaller (both in RAM and disk), faster, more reliable than Ubuntu’s Unity desktop.

DDAC: Disney Data Analytics Conference

Just returned from DDAC, Disney’s data analytics conference in Orlando. Great speakers and content, as usual.

2015-09-18-08-44-06-0629

 

Disney puts us up at a nice place.

 

 

2015-09-17-11-34-40-0614

 

 

Colin Powell gave an inspiring keynote.

 

 

 

 

DisneyPic

 

We had fun events each evening. One of them was a party with all kinds of activities. I went on a galactic mission with 2 of my colleagues, using some of the cool technology from Guardians of the Galaxy.

 

2015-09-17-19-48-12-06202015-09-17-19-48-32-0623

 

 

I met Captain  America and we readied ourselves to defend against enemies of freedom.

 

 

 

2015-09-17-19-18-57-06192015-09-17-20-23-42-0627

 

 

I met Baymax, who is irresistibly huggable.

 

 

 

 

2015-09-18-17-36-19-0631

 

On the last night we finally got time to visit one of the parks, only 2 hours before it closed.

 

 

2015-09-18-18-38-42-0632

 

That was still enough time to ride a roller coaster,

 

 

2015-09-18-19-40-41-0633-new

 

 

visit the Twilight Zone, and fend off an attack from the Empire.

 

 

 

2015-09-19-13-06-56-0637

 

Finally, it was time to return home. Nice view of Mt. Rainier on Sat morning.

 

Ubuntu on 2015 Thinkpad Carbon X1

This week Disney gave me a 2015 Thinkpad Carbon X1 and I set it up to run Ubuntu. I choose XUbuntu because it has a smaller disk footprint than Ubuntu, the desktop uses less RAM and CPU, and I prefer XFCE’s simple old fashioned desktop over Unity. Yet the only difference is the desktop – it is the same Ubuntu under the covers, so it has the most up-to-date repos and is one of the most popular distros, which means good support.

This Carbon X1 is really nice – Disney gives us great hardware. i7 processor, 8 GB RAM, 256 GB SSD, and 1920×1080 non-touch screen. Being a fast typist and keyboard snob, I will say this laptop has the best keyboard I have ever used on a laptop. Better than my Macbook Air, and far better than the terrible keyboard on the Dell XPS-13. And this X1 is one fast machine. It puts my Macbook Air to shame.

The 2015 X1 has Broadwell processors and the longer battery life they make possible. It runs Ubuntu right out of the box, basically works including bluetooth, networking, wifi, even the docking box. But there are some bugs that require tweaking to fix. This is what this blog is all about.

Suspend/Wake

First, I observed that it wouldn’t wake properly from suspend. Upon wake I got a screen that was solid black or garbled.

The first thing I tried was adding the “nomodeset” parmeter to the linux kernel:

sudo vi /etc/default/grub
---
GRUB_CMDLINE_LINUX="nomodeset"
---
sudo update-grub
sudo reboot

This fixed suspend/wake but broke the display brightness function keys. The display was stuck at max brightness. Long story short, for both to work I reverted the above change and did the following:

sudo vi /usr/share/X11/xorg.conf.d/20-intel.conf
---
Section "Device"
Identifier "Intel Graphics"
Driver "intel"
Option "AccelMethod" "uxa"
EndSection
---
sudo reboot

Networking

Sometimes networking won’t connect. This happens wired – ethernet to USB or in the dock, and wireless. Once connected it’s solid; this bug is about initiating a network connection. Disabling and re-enabling networking and wireless don’t fix it. The network settings are often greyed out on the system menu, with no explanation. There’s nothing obvious I can find in the dmesg logs.

It took me a few weeks but I finally figured this out. First use rfkill, which usually fixes it. If not, restart the network manager service. Or just run this script:

#!/usr/bin/env bash
sudo rfkill block all
sleep 2
sudo rfkill unblock all
sleep 2
sudo service networking restart
sudo service network-manager restart
sudo resolvconf -u

Power Management

Ubuntu 15.04 has a feature (I use the word loosely) that cools the CPU by scheduling kernel threads that hog a timeslice while idling. This is redundant and overly complex. There are already 2 ways the laptop cools the CPU: (A) slow down the clock speed, or (B) turn on the fan. Which to use depends on whether you want battery life or performance.

This new feature will have your laptop suddenly become slow and unresponsive at seemingly random times for no apparent reason, and you’ll see mysterious idle threads hogging the CPU.

To disable this blacklist the intel_powerclamp module:

sudo vi /etc/modprobe.d/blacklist.conf
---
# blacklist intel powerclamp, a redundant and overly complex way to keep the CPU cool
blacklist intel_powerclamp
---
sudo update-initramfs -u
sudo reboot

Another action you can take to improve battery life is to enable power save mode:

sudo pm-powersave true

VirtualBox, Windows, and Samba Shares

Disney uses Exchange email, so I created a Windows 7 VirtualBox VM to run Outlook. Thanks to my MSDN subscription, I also have Office 365 but prefer using LibreOffice most of the time.

To access the Linux filesystem from the Windows VM, I create a Samba share in Ubuntu accessible only to my local user and map it to a network drive in the VM. This works mostly fine. But whenever the VM is (re)started, it can take 30 seconds or so before the fileshare is accessible. During this time, Outlook expects to see the drive, gets confused and hangs. It remains unresponsive even after the drive becomes accessible. It must be killed and restarted, which sometimes corrupts the PST file.

I discovered there is a workaround for this. Define the samba share in Virtualbox (in the settings for the VM) instead of in Ubuntu. When I do this, the share is immediately accessible every time the VM (re)starts. No more Outlook hangs and corrupted PST files.

Summary

With these simple changes, my X1 is running XUbuntu 15.04 smoothly and reliably. Even session swapping between users is working! If I encounter any other issues I’ll report them and solutions here.

DNS, Web Hosting and Email

Where’s the old BLOG? It’s here:
Mike’s Old Blog

As is evident from these pages, we have a DNS (mclements.net), host a web site, and email on our DNS. We’ve had this for about 15 years. Started with
Register.com, then used GoDaddy for a few years. The time came to renew and GoDaddy wanted about $145 for 2 years.
Our GoDaddy experience was good, but it had some limitations:

  • Hosting Storage: limited to 10GB unless you pay more
  • Hosting Features: support for WordPress and others cost extra
  • Email storage: about 100MB to share across all accounts
  • Email POP3 only: cost extra to enable IMAP
  • I looked at some alternatives and ended up with Arvixe, as you can see if you ‘whois’ this DNS.

    Arvixe addresses all of the above limitations of GoDaddy:

  • Hosting Storage: unlimited
  • Hosting Features: includes WordPress and others
  • Email storage: unlimited
  • Email IMAP: included
  • The cost is also lower: $96 for 2 years.

    The support is comparable: both respond quickly via email and have 24 hour phone support.

    Of these improvements, I find IMAP email to be the best. We’d been using POP3 for 15 years and got used to the limitations. Key benefits of IMAP include:

    Multi-device: IMAP has one consistent list of which emails you’ve read, stored at the server. When you access the same email account from multiple devices, this list remains consistent. When using different devices, you don’t need to wade through the same emails you’ve already read to find new ones. This is a big improvement over POP3, in which each device separately stores its own list.

    Goodbye, self-BCC: on POP3 if you want to see emails you sent from different devices, you need to BCC yourself else they’ll never appear on other devices. With IMAP, the sent folder is synchronized across all devices so you don’t need to do this.

    Spam management: on POP3 you need to access the email server directly from a browser (not from your email client) to access the folder where it stores Spam, in order to see what’s being flagged as Spam and see if there are any valid emails in there. With IMAP, the server Spam folder is synchronized across all devices so you can access and manage it directly from the email client on any device.

    Synchronized deletes: when you delete emails on IMAP they go to the server Trash folder, which is synchronized just like all other server folders. Thus any email you delete, from any device, goes to the same place. It can be seen and managed directly in the email client on any other device.

    Email refresh: IMAP can use the Idle command to notify clients when emails arrive, instead of having to poll the server periodically like POP3 does.