As part of my effort to help to track down and, ultimately, have the powerline noise in my neighborhood eliminated, I felt it was necessary to collect a lot more data about the problem, especially to help predict when the noise would be most audible so PG&E technicians would stand a better chance of being able to find the source of the problem. (To date, however, they’ve done exactly nothing.)
To that end, I did what I do best: I wrote some code. It turns out to be straightforward in Python to sample audio off a sound card, produce graphs, and upload them with sftp to a web server. The problem of pulse detection was a fun one. My initial approach was to use Fast Fourier Transforms (from the NumPy module) in an effort to look for a 120Hz signal and its harmonics. This approach, however, was ill-suited to the problem. The noise PG&E is making comes in very brief pulses, lasting only 3-5ms, maybe a total of 24-48 samples from beginning to end at 16000 samples-per-second. It turns out this isn’t great for analysis with FFT.
I decided to take a different approach and run my analysis in the time domain instead of the frequency domain. Because what I am looking for is a very predictable and highly consistent exactly 120 pulses per second, it’s possible to create an array of coefficients with 1’s spaced exactly 1/120s apart and 0’s everywhere else. Then this array can be “slid” along the sample array, multiplying the arrays together and summing them for each point. Where the sum is the largest, the alignment is optimized for a repeating pulse at 120pps (because that’s where the most 1s lined up perfectly with the pulses). I also found this method produced even better results with multiple 1’s in sequence; I therefore use a sequence of three 1’s.
Once the best alignment with 120pps can be found, it’s a simple matter to backtrack to the beginning of the sampled audio and run the same analysis again from that point to arrive at a raw sum of the sample amplitude. Finding the noise floor is a similar operation. I take a little bit of a shortcut here and analyze the audio exactly midway between the 120pps peaks. This is an approximation, of course, and it has one significant limitation: the pulses have such a high relative amplitude that they de-sense the receiver to a small extent. This has the effect of making a pulse with a high amplitude cause the noise floor to appear to be somewhat lower than it really is. Another limitation is that a high noise floor can cause the amplitude of the 120pps signal to appear higher; I believe this is because it isn’t truly possible to completely discern a signal from noise; at every point along the time domain input, what we’re actually seeing is a sum of the signal and the noise. To put it another way, the noise doesn’t stop being there just because there’s a signal present. So the “signal” level is really a sum of the signal and the noise floor.
Luckily, for the most part, the noise floor is relatively steady and changes slowly over time. That is, until there’s a solar flare:
Something notable happened to the noise floor just before 6 AM Pacific on April 22 and caused quite a ruckus for the remainder of the day. I happened to notice on Mastodon from the Solar Flare Alert bot reports of significant solar flares right around the time the noise floor went crazy. It’s a fascinating result that I wasn’t expecting when I started on this project.
In this graph you can also see how the signal and noise levels affect each other to some extent, but the key important thing is noting when the signal (the 120pps noise) level exceeds the arbitrary threshold I chose, -48dB. Just by observation, this seems to be about the level where the 120pps noise is significant enough to render the 80m band useless, and it also seems to be about the midpoint between “on” and “off”, which stands out a little bit better in this graph:
The symmetry in question stands out reasonably well in this chart, especially between about 5:30 AM and 8 AM. One thing I’m hoping to find in the next several days or weeks of data is some pattern to when the noise is most likely to be present, in the hope that I can give PG&E’s technicians some idea of when they should come out looking for a problem.
The data is sampled once a minute, and the graph and the signal, noise, and SNR data along with some current weather conditions are recorded in a CSV file; the plot is regenerated, and both are immediately (re)uploaded to the n6ol.us server. Then it’s a simple matter of updating the filename once a day to build up a complete set of charts and recorded data segmented by day. You can view the complete set of files at https://n6ol.us/noise/
Remaining is the challenge of getting PG&E to actually do something about the problem. So far they’ve been ignoring it as hard as they can or denying that it’s even possible for them to generate RFI that affects shortwave radio (which is, of course, nonsense).