Plex Server Setup Notes

We have a collection of DVDs and music CDs that we would like to be able to watch and listen to, but storing them takes space.  Rather than keep the discs around where they are cluttering the room, I am working on loading them into a Plex Media Server.  That way, we can watch the DVDs through the Chromecast attached to our TV and the music, either through the Chromecast on the TV or the Chromecast Audio in the kitchen.

Basic Plan

The Plex_Server holds all the media and streams it do devices or Chromecast.  Chromecast is connected to the projector.  Projector is connected to the stereo for audio.

Media Controller can be the Plex client app running on a cell phone.  We might want to get a dedicated tablet to run the client as a controller so that the boys or guests can control the video without having to use one or our phones or to grant Plex access specifically to them.  Configuration and control also happen through a web app running on the server.

For now, the Plex server will be in the room with us, but I plan to eventually move it somewhere else.  As long as both the Chromcast and the Plex Server have good connections,  they need not be in the same room.  That is, the server can be noisy and
have fans with no need to worry about fan noise disrupting movie watching.

How to Configure

The server will be running headless and unattended, so I am not installing a GUI on it.  Here is what I have done so far starting with a base Ubuntu 16.04 server (no GUI):

  1. Install Plex Server on the server.  I used the [Plex PPA](https://launchpad.net/~plexapp/+archive/ubuntu/plexht) to install the `plexmediaserver` package.
  2. Copy some content to the server in an obvious place.  I had the MP-3 files I previously ripped and loaded onto my MP-3 player to start with.  More on converting content later.
  3. Configure the Plex server through the web interface, http://host:32400/manage/index.html, where host is the hostname or IP address of the Plex Server.
    1. Look for the “Library” configuration item and add a library.
    2. Assign the folder with content to the library.  Plex likes to have is particular about how files are named and where they go.  Every movie that does not follow the naming convention seems to get labeled as Madagascar 3 (at least for me).
    3. See that the content appears in the web browser.
  4. Run the Plex client on a phone/tablet.  See that content is present.
  5. Install MakeMKV using the online instructions for MakeMKV.
  6. Install Handbrake using the online instructions for Handbrake. (Basically, use the Handbrake PPA.)

How to Load DVD Content

Plex can play back content, but does not load it from external media.  We have about 100 DVDs when you count all the “complete collections” of X Files, Monty Python, Firefly, etc.

This method seems to work manually, and needs to be automated to make loading all those DVDs easier.  Also, consider running handbrake on a different computer than Plex Media Server to improve performance (not bogging down the server, and a machine with a
GPU and/or more cores might go faster).  Ideally, a batch queuing system could schedule transcoding jobs as processors come available.

I am not sure if transcoding is worthwhile at this point.  At the highest quality preset, I still see a lot of compression artifacts playing back SD DVDs on an HD monitor.  The highest quality H.264 video is about half the size of the original, and takes over 5 hours to transcode.

Here is my basic manual procedure

0. Make sure the directory exists somewhere and Plex Media Server points to it,
assuming you are processing a movie.  For TV shows, do the same, but put them in the TV directory instead of the Movie directory.  I don’t know where exercise videos go. They are probably more like TV than Movies.

  1. Insert DVD into Plex Server
  2. Make a directory for the content called “Movie Title (Year)“, where Movie Title is the title of the movie and Year is the year in which it is released.  If unsure look for the movie on imdb.com.  That is what Plex will look for to find artwork for the movie.
  3. Load the DVD into Movie Title (Year) with makemkvcon mkv disc:0 all "Movie Title (Year)".
  4. Transcode files in Movie Title (Year) withHandBrakeCLI -Z "Super HQ 1080p30 Surround" -O -i title00.mkv -o Movies/"Movie Title (Year)".mp4
    There might be more than one “title”, especially for TV shows. In that case, repeat running Handbrake for each title.

    The above preset is for high quality, but is very slow.

I still have not worked out the best way to store the DVD “extras” (interviews, clips, cut scenes, etc.) The Plex documentation gives a naming convention for all the extra content, but MakeMKV just dumps the content into files called title00.mkv, title01.mkv, etc. That is, no title or description is readily apparent for each .mkv file.

Resources

Plex Documentation on Loading
Content

MakeMKV Documentation

Handbrake Documentation

Next ToDos

  • Watch movies and make sure quality is OK, usability is OK, etc.
  • Buy Plex Pass (assuming Laura likes it)
  • Are exercise videos more like TV Shows or Movies?
  • Automate DVD ripping. One script to run MakeMKV, assign a sensible name,
    and transcode into the movies directory. Use nice.
  • Figure out CD ripping (manual)
  • Automate CD ripping

Maybe Later

  • Batch queuing for transcoding (cross servers)
  • Containerize ripping/transcoding and/or Plex Media Server for better
    CPU limiting (anything like HTB qdisc available?)
  • Insert disk and server automatically determines kind (CD, DVD Movie, DVD
    TV), rips, transcodes, and deposits the result in the right library.
  • Photos
  • SMB share for media (to allow easier uploading of content)
  • OpenID authentication for friends
Advertisements

Slow Cooker Chicken with Spinach and Kale

This recipe is based on Slow Cooker Chicken Saag Curry.  I have modified it to better match the packaging sizes available in local grocery stores.  Mainly, the spinach, kale, and chicken amounts are increased.  I left the spice amounts at the original values, because we were trying to feed this to our son who was around 2 years old at the time.  You might prefer it with more spice.  After my modifications, it is far enough from any Chicken Saag I have ever had at a restaurant that I translated the name to English to avoid unmet expectations.

I am still experimenting to get the right balance of spinach, kale, and yogurt, but posting here, in case the scrap of paper I wrote the recipe on gets lost.

Ingredients

1 Tbsp Ghee or Oil
1 Red Onion, Chopped
3 Cloves Garlic, Crushed and Chopped
1 Tbsp Grated Ginger
1 tsp Chili Powder
2 tsp Coriander
1 tsp Turmeric
Salt to Taste
1 tsp Garam Masala
1 Cup Crushed Tomatoes
1.25 lb (roughly) Chopped Chicken Breasts (about 6)
16 oz Frozen Spinach (1 bag)
8 oz Frozen Kale (1/2 bag)
(optional) 1 Cup Plain Yogurt

Directions

  1. Brown onion, garlic, and ginger in ghee or oil.  Thaw greens in microwave.
  2. Add spices, except garam masala.  Stir fry several for several minutes.
  3. Add tomatoes and bring to a boil.
  4. Add garam masala.
  5. Add greens.  They need not be fully thawed, but should not be completely frozen.  If they have been thawing since you started Step 1, above, they should be fine.  Stir to mix.
  6. Pour spinach mixture into slow cooker, or refrigerate over night.
  7. Add chopped chicken to slow cooker and combine.
  8. Cook on low 6-7 hours.
  9. Add garam masala to taste 30 minutes before end of cooking.
  10. (optional) 30 minutes or less before serving remove from heat and mix in yogurt.  If the yogurt gets too hot or too hot to fast, it will curdle.  Try adding gradually adding a small amount of liquid at a time from the pot to the yogurt to bring it up to temperature, then pour the yogurt into the pot.  If you plan to reheat some later, keep the saved part separate and add yogurt to it after reheating.  (The yogurt will tolerate some time in the microwave, but not too much.)  Experimentation is required to determine precise limits for this step.
  11. Serve with rice, poppadoms, or naan.

Fixed nameserver breakage on Linux desktop machine

The network configuration on the home desktop/server is unusual. It had previously been configured to share DNS records with another zone. Moreover, the wireless network is statically configured, because it only ever needs to connect to the home Wireless AP, and I want it to connect on boot, so that other hosts can connect in. Network Manager does not want to connect until someone logs in.

The `/etc/resolv.conf` was not being updated with the Wireless AP as the dns server to handle forwarding. The normal instructions assume you are not running a local DNS server (which is typical). To fix DNS, I had to add a `forwarders` entry in `/etc/bind/named.conf.options`. Not rocket science, but I will probably forget by the next time it needs to be configured.

Evolution Predicts Scientists to Go Extinct

Tired of those snooty scientists telling you that you were not designed by a devine being? You can take comfort that their precious theory of evolution predicts that one day they will all be extinct.

The top of the science ladder is a research job at a university or government lab.  Imagine you are wanting to get this job.  All you have to do is go through four years of undergraduate school, and get into a good graduate school.  In graduate school, you will be worked like a dog, first with classwork, and then to produce papers for your mentor, not to mention your own research.

Upon getting a Ph. D., you are now qualified to work for 4-6 years in a postdoctoral position, where you will work like a dog to get research funding and write papers for your mentor.  Only then will you be qualified for a tenure track position at a university, where you will spend the next 6 years working like a dog chasing research funding and writing papers in the hope of getting tenure.

All that time spent in the lab and office does not leave a lot of time to find a mate and raise children.  It would be easiest to wait until after tenure.  A smart, driven scientist, focused on career, might get tenure by age 36, but that path guarantees far more hours in the lab and writing papers and grant proposals.  In 2003, the average age at which science faculty got tenure was 39.  Fertility for both sexes peaks in the 20’s and beings to decline quickly for women over 35.  The decline is slower for men, though studies have found higher incidence of birth defects and more difficulty in getting a woman pregnant for men over 40.

At best, we can expect that scientists will be less likely to conceive children than the general population. When they do have children, they will likely be able to have fewer children, and those children will have higher incidence of birth defects.  At least one parent will have less time to care to spend on child-rearing, and possibly both — after all, scientists are most likely to spend time around, and ultimately mate with, other scientists.

Evolution is all about survival of the fittest, and the evolutionary definition of fitness is ability to pass genes on to the next generation.  To the extent that genetics contributes to ability to do science, and science happens in academia, the current system is removing the genes that lead to high academic performance from the gene pool.  You might argue that a fair amount of science gets done in start companies and research divisions of larger companies.  That is true, but it is also true that many of those people work hours competing with those in academia.

Condition Variables are not Events

I just spent several days chasing down a bug that essentially amounted to someone assuming that a condition variable behaves just like an event.  I have seen this a number of times before, and it is frustrating fixing the same issues over and over again.

Basically, the broken code looked something like this:

mutex mtx;
condition_variable cv;
{
    unique_lock<mutex>lock(mtx);
    unsigned long wait = 1;
    RunOnOtherThread([&cv]() {
        /* do some work */
        cv.notify_one();
        wait = 0;
    });

    cv.wait(mtx);
    do { this_thread::yield() } while (wait);
}

RunOnOtherThread is a method that takes a function as a parameter and runs it on another thread. The current thread has some work it would like done, and having the other thread do it avoids some synchronization issues.

Synchronization with Events

The person who wrote this code expects condition variables to work like events. An event synchronizes threads by indicating that a certain state has been reached. Here is the equivalent of the above code using an event:

Event evt;
unsigned long wait = 1;
RunOnOtherThread([&evt]() {
    /* do some work */
    evt.set();
    wait = 0;
});

evt.wait();
do { this_thread::yield() } while (wait);

The thread that is calling RunOnOtherThread creates an Event object and passes it to the other thread through the capture of the lambda. The event starts out in a reset state. Any thread that calls evt.wait() while the Event is reset waits until the event’s state becomes set, by some other thread calling evt.set(). When a thread calls evt.set(), the event’s state becomes set and stays that way until it is reset (either automatically by reading the state in event.wait() or manually depending on the type of event object).

The event-based example above works with a typical implementation of Event objects. The obvious case is if the code in the outer function runs first, it will run until it hits evt.wait(), then block until the lambda completes. If the lambda should be scheduled to run first (or concurrently), that’s fine — the state of evt will be set and the first thread will pass through evt.wait() without ever blocking.

The C++ standard library does not provide an implementation of Events. It does, however, provide mutexes and condition variables.  So, the programmer substituted a condition variable for an event.

The anonymous programmer has added a custom spin lock at the end of the first thread to guarantee that the all to evt.set() has completed before the current thread goes out of scope and its destructor frees the evt object’s memory.

Mutexes and Condition Variables

A mutex provides mutual exclusion for a region of code. After locking a mutex, a thread is guaranteed that no other thread that has locked the same mutex can run at the same time. If one thread tries to lock a mutex while another thread has the same mutex locked the second thread blocks until the first thread unlocks the mutex.

In the first example above, creating a unique_lock object initialized with mtx locks the mutex, and holds the lock for as long as the unique_lock object is in scope. Wrapping the mutex in an object is more robust than calling explicit lock and unlock functions, because if someone comes a long later and adds a return in the middle of the function, the mutex will automatically get unlocked when lock goes out of scope.

Mutexes only provide mutual exclusion. They do not provide any guarantees about the order in which operations happen. For that we need condition variables. Condition variables are named for their use in implementing monitors. They are designed to help a thread block until a certain condition is true. The basic operations on a condition variable are wait and notify.

A thread waiting on a condition variable atomically unlocks a mutex and blocks until another thread signals the condition variable by calling notify — or until the thread just happens to unblock (there are number of implementation dependent reasons this may happen). Upon unblocking from the condition variable, the thread has re-acquires the mutex. The unblocking thread is responsible for testing a condition to determine whether it should continue or block again.

Not checking the condition that the thread was waiting for is a common bug. The typical correct design pattern for using a condition variable looks something like:

mutex mtx;
unique_lock lock(mtx);
condition_variable cv;

while (!my_condition)
{
    cv.wait(lock);
}

Any arbitrarily expression could be substituted for my_condition. The above code segment will lock mtx, wait until my_condition is true, and then continue executing with mtx locked. To wake up a blocking thread, the thread that makes the condition true must call either notify_one or notify_all. If one or more threads are blocked on the condition variable, notify_one wakes up at least one of them, and notify_all wakes up all of them.

Testing my_condition before blocking on the condition variable is critical, because a thread only wakes up from being blocked on a condition variable when it is blocked on the condition variable at the same time the signal arrives. So, if the lambda in the first example, runs completely before the first thread calls cv.wait() the first thread will block forever waiting to unblock.

It is also worth noting that pretty much every implementation of condition variables does not guarantee that the thread will never awake before being signalled. A loop testing the condition is required, in case the thread wakes up before some other thread signals that the condition is ready. So, it is possible that the first thread in the example could wake up at any time while the lambda is still executing. The spin lock will catch it before it exits, but it requires the first thread to busy wait, taking CPU and memory access cycles away from other threads.

No Spin

The spin lock at the end of the example code is an inelegant way to prevent the original thread going out of scope before the lambda completes. Leaving it out of the original version could cause the program to crash if the lambda is still in cv.notify_one() when cv goes out of scope in the original thread. Here is a cleaner implementation:

    mutex mtx;
    condition_variable cv;
    bool otherDone = false;

    RunOnOtherThread([&cv, &mtx]() {
        /* do some work */
        unique_lock<mutex>lock(mtx);
        otherDone = true;
        cv.notify_one();
    });

    {
        unique_lock<mutex>lock(mtx);
        while (!otherDone) 
        {
            cv.wait(mtx);
        }
    }

In this implementation, the other thread will do its work then set otherDone true to indicate it is done. I then calls cv.notify_one to wake the first thread. Locking mtx guarantees that the changes to otherDone will be seen by the first thread. If the lambda completes before the while loop tests otherDone, the first thread will complete without blocking. Otherwise, the first thread will block until the lambda calls cv.notify_one() and releases the lock on mtx. There is no need to worry about the first thread exiting before the lambda completes, because the first thread cannot unblock from cv.wait() until it acquires the mutex, which is only released when the lambda exits.

Used properly, condition variables can simplify a lot of typical synchronization problems. A good operating systems text book should have examples for producer/consumer, and all the other standard examples.

Of course, all this assumes that one thread calling another to do something and blocking waiting for the result makes more sense than having the first thread simply do all the work. In this case, the original author was trying to avoid complex synchronization issues that arise when lots of threads try to access the same data.

When high concurrency is not critical to meeting system requirements, the simplicity of doing everything in the same thread can save a lot of headaches. In other cases, more parallelism is required to meet performance goals. It all depends on the application.

Fisher-Price Swing Repair

My son sleeps exclusively in a Fisher-Price Cradle ‘n Swing.  The swing stopped swinging during the night last night, so I took it apart to see if I could fix it before tonight.  The swing is still in pretty good shape, though it is an older model we bought second hand from Once Upon a Child.

When I got the top open, I could smell the odor of burnt electronics.  Fortunately, the bad transistor did me the favor of leaving a scorch mark to let me know what needed replacing.  The scorch marks are a little hard to see in the first picture, but pretty obvious with the transistor clipped off.

Fisher-Price swing control PCB with fried transistor Fisher-Price swing control PCB with fried transistor removed to show scorch marks

The markings on the part indicate that it is most likely an ON Semiconductor MJE171G PNP transistor.  Since we needed the swing tonight, I headed out to the local Radio Shack to see what they have on hand.

There I found a $2 Tip 42 PNP transistor with comparable specs.  The only spec that is not up to the original MJE171G is the maximum emitter-base voltage — 7.0 V for the MJE171G, but only 5.0 V for the Tip 42.  When I probed the drive circuit without a transistor installed, the maximum emitter-base voltage was less than 4.3V for any setting of the motor speed dial, so the Tip 42 will hopefully be fine.  The picture below shows the Tip 42 installed in place of the MJE171G.

Fisher-Price swing control PCB with replacement Tip 42 transistor     Fisher-Price swing control PCB with replacement transistor

The motor still spins when power is applied, but needed a little push to get going.  Rather than to save a second trip to the store for a motor, I also got a replacement motor while I was out.

A helpful Instructables page called Repair your FisherPrice cradle swing describes how to re-purpose the motor from an Air Wick air freshener as a replacement for the Fisher-Price swing motor.  The local Meijer store had one for $5.

It is interesting that the control PCB pictured in the Instructables article has a TO-220 transistor with a big heat sink prominently mounted on a PCB.  It looks like our, apparently older, swing must have had enough transistors blow out that the redesigned with a beefier transistor and better cooling.  Had I read the article all the way through and seen the size of the heat sink on the newer model, I would have bought a heat sink, too.

After replacing the transistor and motor, the swing is back in working order.  I am sure I could have bought the parts online from Digikey or Mouser for much less, but shipping would have eaten up the difference.  Even more importantly, the total price was low enough that getting all the parts fast enough to repair the swing before bedtime was more worthwhile than saving a few dollars.

Configuring a Gateway M-6750 for NDIS Wrapper on Ubuntu 12.10

The hard drive my wife’s laptop died recently, so I took that as an opportunity to switch it from Windows Vista to Linux, specifically Ubuntu 12.10.  The only problem I ran into after doing a default install was that the wireless chipset is not supported out of the box. The Ubuntu documentation for setting up wireless with (and without) ndiswrapper is generally pretty good, but I needed a few bits of extra info to make wifi work.

The main problem is that the Gateway M-6750 uses a Marvell 88W8362E wifi b/g/n chipset and comes with Windows Vista installed.  There is no open source driver, so I had to use ndiswrapper.  I went to Gateway’s web site to look for a Windows driver when I got to the step to download the Windows driver (see “Installing Windows driver“). Gateway posts a Windows Vista driver for the wireless chip, but no Windows XP drivers. Unfortunately, ndsiwrapper cannot yet load Vista drivers. It only loads Windows XP drivers.

After a number of web searches, I found a reference suggesting that the Windows XP Gateway M-153 driver uses a similar chipset to the M-67 chipset.  The M-153 driver is the one I eventually got working.

The trick not included in the Ubuntu community install docs is how to get the driver files out of the downloaded zip file. As described in a forum question, I had to do the following:

$ mkdir Marvell   # Make a directory to hold unzipped driver files
$ cd Marvell
$ unzip ../M-153_WIRELESS_MARVELL_2.1.4.6.zip
$ wine Setup.exe

I did this on another Ubuntu machine with wine already installed, but if should work on the M-6750, assuming all the right packages are installed — see the forum thread for details.  Running Setup.exe in wine will pop up an install wizard for the driver.  Go as far as possible through the install process.  It will not succeed, but it will put the files that you need for ndiswrapper in ${HOME}/.wine/drive_c/Program Files/Marvell/TopDog Driver.

The two necessary driver files are NetMW14x.inf and netmw145.sys. Copy those files to the M-6750 (if you are not already working there). From there, I continued with the install Ubuntu community instructions.

When trying to install the driver with modprobe, modprobe returned an error saying that the ndiswrapper module could not be found (even though the ndiswrapper package was installed). An askubuntu.com post indicated that the problem was the 1.57 version of ndiswrapper shipped with Ubuntu 12.10. Compiling and installing 1.58rc1 from source fixed the problem.  Hopefully, ndiswrapper 1.58 will be released soon and Unbuntu 12.10 will be updated soon.

With ndiswrapper 1.58rc1, the remaining instructions worked to get the driver installed.  A March 2010 blog post by Joe Wein describes other problems setting up a Gateway M-6750, but I did not experience them, most likely because of updated drivers and a better install process in Ubuntu 12.10.