- This topic has 6 replies, 3 voices, and was last updated 1 year, 3 months ago by
.
-
Posts
-
from the ARRL…..
ARRL Systems Service Disruption
05/17/2024Updated 5/17/2024Some members have asked whether their personal information has been compromised in some way. ARRL does not store credit card information anywhere on our systems, and we do not collect social security numbers. Our member database only contains publicly available information like name, address, and call sign along with ARRL specific data like email preferences and membership dates.
Original story below:
5/16/2024We are in the process of responding to a serious incident involving access to our network and headquarters-based systems. Several services, such as Logbook of The World® and the ARRL Learning Center, are affected. Please know that restoring access is our highest priority, and we are expeditiously working with outside industry experts to address the issue. We appreciate your patience.
This story will be updated with new developments.
————————————————————–
Comments from Your ARRL North Texas Section Manager follow:
This is all the league is telling us.
I have been told exactly what is above.
I have been reading a considerable amount of Feedback from our members
on the ARRL-AWARDS Email Talk Group, in summary the majority of members are not very happy with this News “Update” from the league.
Some Members on the reflector are saying, “so big deal LOTW is down, get over it”My issues with this are:
The news release is not concise as to what happened and what specific issues there are which need fixed.
If there has been a Data Breech they are responsible to inform their members, sooner than later.Fact is what we know is more found on our own….
For anyone that uses ARRL LOTW on a regular basis: (LOTW is Logbook of The World)
We LOTW users already know LOTW is NOT accessible, we can neither view our Data or Update it.
The same issue exist for access to anything within The ARRL Learning Network.Compounding the issue is the ARRL has not had an IT Department Director since mid 2021
That IT Manager was considered by most well qualified for the position. However, he was only there Six months, before the CEO
ran him off, because our Information Technology Manager Michael Keane K1MK, was exhausted from the way the CEO was Micro Manages Employees
within his department (as well as all others).Which in my Opinion is:
The Information Technology Structure within our Organization (ARRL) is Broken, because we do not have an IT Manager.
Which is why the League is now hiring outside resources to Diagnose and Fix this.My gut says that the League is not telling us the whole story on this
I heard from a well know source that they asked a league official at the Xenia (Dayton) Ham Radio Convention yesterday about
LOTW being down, the reply was “I can not elaborate beyond, It is Not good, in fact it is bad really bad”.Personally I am keeping an extra close eye on my Credit Card reports and banking info.
Meanwhile we are seeing More degradation to the Services we ARRL Members are Paying for.
I use LOTW daily when I am on the Air, it is the second most reason why I continue to be an ARRL Member.
The number one reason I am a member is because I wish to continue to serve my fellow ARRL members as an Elected Official.Steve
KG5VKAnd we will not see an ETA on the fix.
It should be Noted it is NOT just LOTW that is effected.
This is a Malware or a Ransomware attack by the indication of what they have inferred in the limited news releases.
The Learning Center Network as well as tools used within ARRL Headquarters for producing Code Practice Bulletins and several other things
have been impacted.Keep in mind there has not been an IT Manager at the League since Mid 2022 when The Current CEO
Ran that IT Manager off after that IT manager had only been there six months.By all accounts he was a very good IT manager and a well recognized Amateur Radio OP
Steve
KG5VKI believe the ARRL should have posted something like this……
The following is from the Oklahoma Section Manager
Mark Klein, he is a well respected SM and his full time Job is in IT with a Large Paper company.SM044 – ARRL Digital Infrastructure Issues
May 24, 2024
You may have heard of some issues the ARRL has with its digital network infrastructure. This message won’t contain any new information, however, based on the public reports provided on https://arrl.org/, I will try to summarize what’s out there. You may know my training and forty years of “day jobs” are in Computer Science and IT Management. I’ve seen many of these situations and found that these are typically temporary issues. Let’s call them digital speedbumps. Just like driving through a suburban street with physical speed bumps, going too fast can cause additional problems when encountering the next bump.From the ARRL disruption page, I know this is a network-related computer system outage. While the IT Manager in me wants to see how this started and what devices are involved, none of that matters to those on the outside. I also read that the Logbook Of The World database is secure but offline. That’s good news for the “big picture” of us external users. The ARRL Learning Center is also offline. There isn’t any status update on that system. Finally, they have asserted that no personally identifiable information (PII) is stored on-site. Should any information have been shared with external sources, we would be comfortable knowing any PII would haunt this issue. Job 1 for this rebuild is to find, obtain, and verify the required user and system data. Fortunately, the website https://arrl.org is fully functional.
While there may be structured locations like file servers and databases where users should store their data, most users have bad habits of storing information on their local systems. Sorting through all of these storage locations can complicate any recovery. Then, the old infrastructure must be “burned down” and rebuilt. In today’s modern society, much information is required to run the world. Remember that this is a business, and things like payroll, banking, and accounts payable must start back up quickly so employees and vendors get paid.
These events highlight the need for a business continuity plan for your business and even your personal information. Should something like this happen at your house, would your information be retrievable? Are your QSO logs (and other vital data) stored safely and off-site?
The ARRL has a web page dedicated to this situation. As time passes, you should see additional updates here: https://www.arrl.org/news/arrl-systems-service-disruption. Remember that discussions may be limited due to “bad-guy awareness,” future prosecution, or lack of new information.
Rest assured, your Oklahoma and Texas Section Managers and the West Gulf Division Directors are as engaged as we can be. If you are not in Oklahoma or Texas, please contact your Division Directors https://www.arrl.org/divisions to ensure they are working to mitigate this issue.
This is a reminder that “When all else fails, amateur radio is there.” We are 160,000 hams that are still connected by the ionosphere. Grab your microphone, CW key, or FT8 mouse, and get on the air. (Hang on to your QSO logs and update them when the dust settles.)
A link to the page where Mark has this posted is here….
https://ok.arrl.org/sm044-arrl-digital-infrastructure-issues/
Mark is one of Four Section Managers within our ARRL West Gulf Division.
Cheers!
Steve
KG5VKAll,
I just sent the email below to the ARRL board of directors in my capacity of ARRL member at large:
All,
The last time I wrote to all of you was prior to a board of directors meeting. At that time I excoriated the leadership supporting a misguided set of changes that you were considering. Well, I must have struck a nerve since the item was tabled. Was it tabled to kill it I hope? Or just to let some time go by and hopefully the members incensed by the move would lose interest? I plan to make sure they stay interested until it is completely gone.
Instead of engaging in these shenanigans I submit the leadership should be asking the “CEO” why he can’t figure out how to hire an Information Technology person to tend to our business? What has it been, two years since he ran the last one off? If you have an iPhone how many iOS updates have there been in the last two years? You get my point. The website is pretty poorly done. There are a lot of dead links and many other issues. The site was recently compromised. The list of failings is lengthy. If Minster can’t figure out who to hire then I suggest you look to an outside source. And maybe for a new CEO.
ARRL’s site hack and lack of member access to Logbook of The World lays right at Minster’s feet. For the life of me this member can’t understand the lack of accountability. Please stop with the nonsense and start holding Minster responsible for his shortcomings…or replace him.
Best Regards,
Phillip Beall (W5EBC)
If you are an ARRL member please feel free to use this list and write to them yourself:
Rick Roderick (K5UR <K5UR@arrl.org>; Bob Famiglio (K3RF <k3rf@arrl.org>; Marty Pittinger (KB3MXM <kb3mxm@arrl.org>; Carl Luetzelschwab (K9LA <k9la@arrl.org>; Brent Walls (N9BA <n9ba@arrl.org>; Bill Lippert (AC0W <ac0w@arrl.org>; Lynn Nelson (W0ND <w0nd@arrl.org>; David Norris (K5UZ <k5uz@arrl.org>; Ed B. Hudgens (WB4RHQ <wb4rhq@arrl.org>; Scott Yonally (N8SY <n8sy@arrl.org>; Roy Hook(W8REH <w8reh@arrl.org>; Nomar Vizcarrondo (NP4H <np4h@arrl.org>; Art Zygielbaum (K0AIZ <k0aiz@arrl.org>; Dave Propper (K2DP <k2dp@arrl.org>; Fred Kemmerer (AB1OC <ab1oc@arrl.org>; Phillip E. Temples (K9HI <k9hi@arrl.org>; Mike Ritz (W7VO <w7vo@arrl.org>; Mark Tharp (KB7HDX <kb7hdx@arrl.org>; Kristen McIntyre (K6WX <k6wx@arrl.org>; Anthony Marcin (W7XM <w7xm@arrl.org>; James Boehner (N2ZZ <n2zz@arrl.org>; Bill Morine (N2COP <n2cop@arrl.org>; Jeff Ryan (K0RM <k0rm@arrl.org>; Dan Grady (N2SRK <n2srk@arrl.org>; Mickey Baker (N4MB <n4mb@arrl.org>; Jeff Beals (WA4AW <wa4aw@arrl.org>; Richard J. Norton (N6AA <n6aa@arrl.org>; Edward Stearns (AA7A <aa7a@arrl.org>; John Robert Stratton (N5AUS <n5aus@n5aus.com>; Lee H. Cooper (W5LHC <w5lhc@arrl.org>
“Minster” in the note above is David A. Minster (NA2AA). He fancies himself a big deal. To me he appears to be little more than a political hack that some on the ARRL board want there because he is their buddy. Those same members of ARRL leadership appear to me to spend more time playing political games than they do tending to their member’s business. There is plenty of work to be done without ANY political games.
Please feel free to share this with your ARRL member ham friends. Someone needs to smoke the rats out of the wood pile. Having a competent IT professional running your IT operation is an imperative. If the “CEO” of ARRL can’t figure that out after two years, he isn’t CEO material.
One guys opinion.
Phillip
Steve,
I take no solace in reading American Radio Relay League cyberattack takes Logbook of the World offline on the BLEEPINGCOMPUTER.COM website. But, it reinforces my impression of the absolute incompetence of the current ARRL CEO, that the IT director, manager, whatever, position has gone unfilled for two years. To me it looks like he ran the last one off because he wasn’t a boot licking lackey. If I am wrong then maybe someone can elucidate me. I am 100% results oriented and this situation aggravates me beyond belief.
IT security is an increasingly important matter. Literally, inattention to detail on the topic can destroy a business. How far back do they have uncompromised backups? Was it a ransomware attack and if so do they have insurance coverage to help deal with what will be a VERY expensive fix? My guess is that an organization as large as ARRL will have to pay seven figures to fix it, if they have adequate backups to allow for a rebuild. Otherwise, they may be looking at writing a check to some bad actors. To me this is a fireable offense on many levels.
Some on the ARRL board have dithered while their choice – their choice – of CEO left this critically important staff position unfilled. Arguably, the IT head is more important than the CEO of many companies. The ARRL board better wake up and smell the coffee and start making big changes. If they don’t, their membership may start replacing them. Fair warning.
Phillip
While ARRL LOTW (Logbook of The World) has been back up now for weeks.
The struggle at the ARRL to get many other services back online continues
without any given estimates of expected time line for this to be finished.The CEO attitude is members don’t need estimates.
The “Team” is working as fast as they can.Here is a recent video about it from The South East Division Director Micky Baker
https://www.youtube.com/watch?v=DMjwQIsybAQThe ARRL CEO the day after the video above was widely spread via Club sites and individual Blogs
Posted a rebuttal to most of what was said by Mr. Baker
His rebuttal is posted at the bottom of my post here and is of course available on the ARRL websiteIt should be noted I have not always been a fan of Mr. Baker
However, I will side with him versus our current ARRL CEO.Few Notes:
While LOW is usable by members, No DXCC awards or Certificates may be issued
Until the DXCC Database is back on LineARRL membership Stats Database has not been updated since April 2024
This is the Database that allows Leadership to track the health of membership numbers
From Leadership at ARRL in Newington to those of us in the Field (Division Directors and Section Managers)The ARRL Finance Database is Down, all financial transactions are being done manually !
Therefore No Budget Reports to Division Directors as well as Section Managers on what re monthly Budget allowances areThe ARRL field Services Database just this week came back online so Section Managers can
now post request for changes to those in ARES and other areas
So that there is continuity of who is who in the respective areas.
The back log of getting these updated may take a few weeks, possibly more
Again the Time lines are unknown and if know at HQ they will not be shared per the CEO statements.Yes there is still value in being a member of the ARRL and supporting it
Let’s chat about it here or in person
I plan on attending the RRVARC meeting the 28th of this month (September)Cheers!
Steve
KG5VK
ARRL NTX SMARRL Systems Service Disruption
09/25/2024Updated 9/25/24…….
(most of the following but Not all is from the CEO) –
Comment above in parenthesis by Kg5vk
Two major systems that have not been returned to service include DXCC and our accounting system. Most other systems have returned to full service, and we are working to clean up the stragglers.
DXCC® System
Our DXCC software was written some 20 years ago. Although an attempt was made 8 years ago to reprogram it to function in a newer environment, that effort was unsuccessful, so the old system remains running today. The complication with bringing it back to service is that it runs on an older version of Windows that is not supported, and more importantly, we cannot purchase adequate protection to return it to service on an internet-based network.
We have created an air-gapped network to run the system on and have it operating in a test, using a different method for connecting it to Logbook of The World® and online DXCC — the web service that runs the online DXCC application. This configuration is being tested and we expect it can go into production soon. We will work to process the backlog as quickly as possible.
New DXCC?
There were statements made by an ARRL Director about work on a new DXCC system. That is not factual. While we have ideas for merging DXCC and LoTW® into a single ARRL Radiosport platform, this concept has not been fully spec’d out or put into development.
ARRL Financials
The system we use for accounting was hit hard by the cyber-attack. Currently, the system is fully operational and the ARRL Foundation books are in place. ARRL’s corporate books are current as of 6/30/23, and we are working to load the balance of the data through 12/31/23. Once we are certain that the books tie to the audited 2023 financials, we will move to load the 2024 transactions, which are in digital and paper formats. That will take some time, but we will have all the data current in time for the 2025 Annual Meeting and the annual audit.
Backups
Another statement made was about ARRL’s backups of our IT environment. There was an effort last year to move all of our backups into the cloud. That was completed mid-2023. There was a routine discipline of backups that went through mid-May 2024 when the cyber attack occurred. Unfortunately, the entire cloud infrastructure was penetrated, and all of the environments were deleted. Not encrypted. This caused us to revert to other backups that had been made and stored in other places. As a result, we were largely able to continue all of our major services especially publishing our 4 magazines and keeping book projects, like ARRL Handbook 101, on schedule.
Today, ARRL is using a proprietary system to backup systems and data locally and to the cloud. We are also continuing to use air gapped, off-site backups in specific areas and are expanding that for disaster recovery purposes. The new IT environment is currently being rolled out, so the assertion that there is something wrong or staff are incompetent is not factual.
We appreciate your continued patience and understanding.
This story will be updated with new developments.
- You must be logged in to reply to this topic.