Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 15, 2021 12:20:47 GMT
Ah, OK. If you dont use it, you dont use it. You might find it lurking on some internal systems, tho.
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 15, 2021 12:24:08 GMT
I guess different ways of looking at it. Most of us seem to be approaching it from an overall internal infra perspective rather than looking at one specific product, and yeah, it's surprising just how far reaching it is... Like I say, its not specifically called out in a lot of documentation. If you install Product A, it wont necessarily call out the fact that it leverages Product B, C and D, which is what initially made our lives difficult. We have got better handle on it now and are moving into more of a monitoring phase.
|
|
Psiloc
Junior Member
Posts: 1,567
|
Post by Psiloc on Dec 15, 2021 12:38:04 GMT
Our biggest risk is that it gets into their devices via some other software or OS vulnerability that is outside our remit, but because they happen to use the device primarily for our application they're going to point the finger at us.
Luckily we only provide apps to clients that have their own IT department so our position is that's their problem (but in kinder words)
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 15, 2021 13:31:29 GMT
Our scans have all finished and the only thing we have a vulnerable instance of log4j2 on is some ManageEngine products (Quelle Surprise!), which have all been disabled and never had external access in the first place (which is a bonus). If folks are interested, this is a good diagram (and blog post) of how it actually works: From www.govcert.ch/blog/zero-day-exploit-targeting-popular-java-library-log4j/The worst thing I'm having to do now is constantly say to our group top bods "we just have to wait for X (vendor) to confirm their systems are vulnerable or patch their systems if they are". The response (several times) has been "can't we ask them to hurry up?". Yes, of course. And presumably, you think there's a reason why our 350 person company and 50 seat license takes precedence over everything else? Softcat have helpfully put together a list of external vendors, too: www.softcat.com/apache-vulnerability
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 15, 2021 13:37:46 GMT
I see that diagram in my sleep now
|
|
|
Post by dfunked on Dec 15, 2021 13:39:10 GMT
I love that "do they know who we are?!" attitude that execs seem to have.
Yeah, fucking nobody in the grand scheme of things...
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 15, 2021 13:40:25 GMT
I see that diagram in my sleep now Yeah, me... wait. You've been sleeping? Bastard.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 15, 2021 13:42:26 GMT
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 15, 2021 13:46:42 GMT
I love that "do they know who we are?!" attitude that execs seem to have. Yeah, fucking nobody in the grand scheme of things... "But we pay them literally POUNDS a year!" Fuck. Off.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 18, 2021 8:21:29 GMT
|
|
|
Post by dfunked on Dec 18, 2021 8:54:04 GMT
We've got vendors who are insisting that their latest release covers "the vulnerability"
You mean the second one that just got bumped up to a 9 too, not just the first one, right? Right?...
*radio silence*
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 18, 2021 10:06:00 GMT
Innit. I've turned off two servers this morning. 😑
|
|
|
Post by barchetta on Dec 21, 2021 0:07:56 GMT
Seems we still made use of log4j v1.x and the infosec team are happy with that state of affairs. May not be explicitly mentioned in this latest vulnerability case, but I'm surprised we are not recommending v1.x gets updated any which way....
I'm not a programmer/security specialist by any stretch. Just raised my eyebrow at this calmness of their approach to this. Feels like the thought is v1 is so old as to be safer than the affected v2 builds... as if it is like an old self contained system unreachable by bad actors.
|
|
Psiloc
Junior Member
Posts: 1,567
|
Post by Psiloc on Dec 21, 2021 9:17:29 GMT
Apparently some versions of SQL Server ship with an affected version of log4j. It may only be in use for Java based ODBC connections ("JDBC", which I've never used or heard of before in my life) but the .jar file is here:
C:\Program Files\Microsoft SQL Server\150\DTS\Extensions\Common\Jars\
Starting to be pinged by our clients own scans, so the next few days are going to be fun
|
|
Psiloc
Junior Member
Posts: 1,567
|
Post by Psiloc on Dec 21, 2021 16:52:03 GMT
Currently trying to come up with a solution for a customer who has proudly gone paperless and removed all of their printers, but it turns out would still quite like a physical document be produced by our software. The accepted solution is to share a printer in another room and map their other PCs to it and they'll just walk over and fetch it from now on. Problem solved!
I can tell you're wondering who gets to spend his day conducting this essential exercise tomorrow
|
|
cubby
Full Member
doesn't get subtext
Posts: 6,403
|
Post by cubby on Dec 21, 2021 18:20:00 GMT
Don't mind me, just listening to all this log4j talk, wondering what any of it means
|
|
|
Post by Aunt Alison on Dec 21, 2021 18:33:30 GMT
I like reading it to try and work out who has the biggest tech balls
Currently my ranking is nick > dogbot > defunkd > Psolic > barchetta
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 21, 2021 21:07:55 GMT
Seems we still made use of log4j v1.x and the infosec team are happy with that state of affairs. May not be explicitly mentioned in this latest vulnerability case, but I'm surprised we are not recommending v1.x gets updated any which way.... I'm not a programmer/security specialist by any stretch. Just raised my eyebrow at this calmness of their approach to this. Feels like the thought is v1 is so old as to be safer than the affected v2 builds... as if it is like an old self contained system unreachable by bad actors. They are correct. Version 1.x is unaffected because it was never updated with the code in which the vulnerabilities discovered lie. It's not possible to use the exploit against that version of log4j. Usually, although there's quite a lot of good things to be said about not running production systems closer to the bleeding edge than latest -2 or -1, security by obscurity is not the one... but in this case, it actually seems to work. Nessus, for example, recommends upgrading versions below 2.0 to the latest but only because "they may be vulnerable". AFAIK, quite a lot of people still use old versions of things like Apache Struts (for example) because there were significant code changes required between versions 1 and 2.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 21, 2021 21:16:05 GMT
Don't mind me, just listening to all this log4j talk, wondering what any of it means Be very happy. I don't have Nick's levels of infrastructure but I have been doing 18 hour days managing this shit, including all weekend the last couple. Further vulnerabilities are still expected, too. Obviously, it's under scrutiny like never before and everyone is trying to find holes. It's... inevitable other things will be found in addition to the three so far.
|
|
|
Post by Sarfrin on Dec 23, 2021 16:47:33 GMT
Bloody hell. I had no idea the amount of problems it's caused.
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 23, 2021 16:54:11 GMT
I don't have Nick's levels of infrastructure but I have been doing 18 hour days managing this shit, including all weekend the last couple. Fortunately, that level of infrastructure paradoxically means everyones job is a bit easier because we have to have a guy for this, a guy for that and a guy for the other. We are regularly audited and bollocked if we have one man bands. Im an incident lead and lead cyber forensics specialist, we have a sec eng team, a CTI team and my team is in the larger monitoring team and we have had to have all hands on the pump to lab IOCs and then push alerts based on those out. Basically we had to come up with a way to say 'compromise has happened' and monitor for it. Its other peoples job to do the vulnerability assessments, patching and the thousand other jobs that have come with mitigating this shit.
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 23, 2021 16:58:59 GMT
That isnt to say it hasnt been a pain in the arse but our work is, hopefully, largely done so we can clock off in fairly good conscience (I am on call tho).
Sucks to be vulnerability management, though.
|
|
|
Post by 😎 on Dec 23, 2021 17:08:57 GMT
I’m very much enjoying* the near constant flow of “good morning, this is to confirm our products are not affected by whatever new twist to this vulnerability that just happened” from our vendors. *
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 23, 2021 17:12:55 GMT
Again, we have a third party management team who are liaising with all that bullshit.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 23, 2021 17:38:18 GMT
Yeah, so we're a much smaller operation and I pretty much do all of that. As well as hearding all the Ciscos and doing a load of SysAdmin stuff, too.
It's been pretty hectic.
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Dec 23, 2021 17:41:35 GMT
I really don’t envy you. I’ve worked jobs where you have to wear a lot of hats and something always slips through. I do prefer a bit of a narrower lane because it does help retain focus.
If I was on the hook for detection AND prevention I would have lost my fucking mind
|
|
|
Post by 😎 on Dec 23, 2021 17:50:45 GMT
I get them as a courtesy mostly. I have zero dealings with any on-prem or virtual infrastructure (everything I deal with is XaaS) so I’ve escaped this very unscathed.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Dec 23, 2021 18:18:55 GMT
I really don’t envy you. I’ve worked jobs where you have to wear a lot of hats and something always slips through. I do prefer a bit of a narrower lane because it does help retain focus. If I was on the hook for detection AND prevention I would have lost my fucking mind I don't have much left to lose. We have a third party looking after our primary, cloud based application and it isn't affected anyway. So that helps. But I'm getting close to the point where I need a more senior role (ie, managing rather than engineering) or to be getting paid a lot more for the skills I'm using now. I LIKE my job but things have moved on substantially since I joined and there's no danger of payrises. Underpaid and overworked. So, standard IT really.
|
|
dogbot
Full Member
Posts: 8,738
|
Post by dogbot on Jan 14, 2022 12:07:30 GMT
Ready for a good laugh?
My organisation bought a couple of companies in a similar market space last year, ostensibly because we wanted their products.
Instead of doing proper IT due diligence at the time, my colleague and I have been tasked this week with a check of the platforms of one of these companies against Minimum Standards, with a view to bringing them up to spec.
They're primary production systems are on Windows 2003. I shit you not.
Their SAN is a pre-Lenovo IBM and hasn't been patched since 2014.
Their switch stacks are from 2001. VMWare is on v5.
This is going to cost about half a million quid to sort out.
Who cares about due diligence at the time of purchase, eh?
|
|
Bongo Heracles
Junior Member
Technically illegal to ride on public land
Posts: 4,661
|
Post by Bongo Heracles on Jan 14, 2022 12:10:59 GMT
This is why RBS kept shitting the bed a few years ago. They kept buying other business and sticking them to their core systems with blu-tac.
Our integration projects are fucking horrendous.
|
|