It was to talk about “team restructuring”

  •  1984   ( @1984@lemmy.today ) 
    link
    fedilink
    14010 months ago

    Companies are often insane. I’m working in one who has this one guy build a super complicated architecture, because he don’t know aws. So instead of just using a message queue on aws, he is building Java programs and tons of software and containers to try and send messages in a reliable way. Costs the company huge money, but they don’t care, since he is some old timer who has been there for like 10 years and everyone let’s him do what he wants.

    • I personally always try to engineer away from cloud services. They cost you ridiculous amounts of money and all you need is documentation afterwards. Then it can be easier and faster than AWS or GC

        • Got to agree with @Zushii@feddit.de here, although it depends on the scope of your service or project.

          Cloud services are good at getting you up and running quickly, but they are very, very expensive to scale up.

          I work for a financial services company, and we are paying 7 digit monthly AWS bills for an amount of work that could realistically be done with one really big dedicated server. And now we’re required to support multiple cloud providers by some of our customers, we’ve spent a TON of effort trying to untangle from SQS/SNS and other AWS specific technologies.

          Clouds like to tell you:

          • Using the cloud is cheaper than running your own server
          • Using cloud services requires less manpower / labour to maintain and manage
          • It’s easier to get up and running and scale up later using cloud services

          The last item is true, but the first two are only true if you are running a small service. Scaling up on a cloud is not cost effective, and maintaining a complicated cloud architecture can be FAR more complicated than managing a similar centralized architecture.

          • You are paying aws to not have one big server, so you get high availability and dynamic load balancing as instances come and go.

            I agree its not cheaper than being on prem. But it’s much higher quality solutions.

            Today at work, they decided to upgrade from ancient Ubuntu version to a more recent version. Since they don’t use aws properly, they treat servers as pets. So to upgrade Ubuntu, they actually upgraded Ubuntu on the instance instead of creating a new one. This led to grub failing and now they are troubleshooting how to mount disks etc.

            All of this could easily be avoided by using the cloud properly.

            •  wim   ( @wim@lemmy.sdf.org ) 
              link
              fedilink
              10
              edit-2
              10 months ago

              I used to work on an on premise object storage system before, where we required double digits of “nines” availability. High availability is not rocket science. Most scenarios are covered by having 2 or 3 machines.

              I’d also wager that using the cloud properly is a different skillset than properly managing or upgrading a Linux system, not necessarily a cheaper or better one from a company point of view.

          • I worked in operations for a large company that had their own 50,000 sq ft data center with 2000 physical servers, uncountable virtual servers, backup tape robots, etc… Their cooling bill would like to disagree with your assessment about scaling. I was unpacking new servers regularly because, when you own you own servers, not only do you have to buy them, but you have to house them (so much rented space), run them, fix them, cool them, and replace them.

            Don’t get me wrong, I’ve also seen the AWS bill for another large company I worked for and that was staggering. But, we were a smaller tech team and didn’t require a separate ops group specifically to maintain the physical servers.

            • If you really need the scale of 2000 physical machines, you’re at a scale and complexity level where it’s going to be expensive no matter what.

              And I think if you need that kind of resources, you’ll still be cheaper of DIY.

      •  1984   ( @1984@lemmy.today ) 
        link
        fedilink
        67
        edit-2
        10 months ago

        It’s a different form of lock-in since it’s just his creation. When he leaves, all of this will be very hard to maintain and the company will probably rebuild it all on aws.

        I have been bringing this up but they say that it’s too late to change direction now (they are afraid to upset the guy).

        But I’m looking on the bright side. I get to learn a lot of stuff I otherwise I wouldnt if this was a single managed aws service. I’m bringing in terraform and instead of just putting a message queue there, I need to spin up entire architectures to run his ec2 instances with all the apps and everything required to make things work.

        Takes months… So for me it’s fun. I don’t have to pay for it. But companies are crazy. :)

          • It also depends on the age of the company.

            My current company is comparatively young and only really grew above the 100 people mark a few years ago. There are people who only worked here for 10-15 years, but are so integral as head-monopoly, that they might as well have been there forever.

            In my old company, there were developers retiring that worked literally their entire lives for the same company.

            • True, true…

              Aside: Back in my day, we could use the term “relatively” to mean “in relation to” some other thing. Over time it became “in relation to the average thing” instead of a specific thing. Now it just means “a little bit”/“sort of”. Now people use “comparatively” to convey what “relatively” used to mean. Except… you just now seem to be making that same “relatively” transition with the word “comparatively”. I just find language interesting, and wonder what the next “relatively” will be once that meaning has been lost even to “comparatively”.

                •  NightAuthor   ( @NightAuthor@beehaw.org ) 
                  link
                  fedilink
                  English
                  4
                  edit-2
                  10 months ago

                  no no no, its not a critique specifically of you. Native english speakers do this all the time. And I’m sure its inevitable that “comparatively” will make that transition too.

                  I’m interested: is there a german word to replace "vergleichsweise " to more explicitly mean “comparison”?

    • What the company likes about the old timer is that because he has been there for 10 years, he will likely be there for the next 10 years to support the complicated system he is creating now. If a younger team member creates something using a modern approach, there is the risk they will leave in a years time and no one knows how the system works.

        • Oh god. I do a lot of PowerShell scripting at my place, and less than half my team is proficient in it. My co-workers who are almost never write comments in their scripts. Meanwhile, if it’s anything that will live longer than ~5 manual runs, I spemd more time on comments and documentation than scripting.

          That effort is valued, but I’m shocked that my team isn’t more aware of the need for documentation. We literally experienced the “bus factor” situation a few years ago.

    • That describes two out of the four jobs I’ve had in my career lol

      It pays the bills though so what can I say, the tech I actually want to work with is what personal projects are for lol

  • “Team restructuring” is so much fun, you never know what you’re going to get.

    Your boss’s boss now reports to a slightly different VP? Everyone is getting fired? No way to know which it’s going to be, until the end of the meeting.

  • That happened to me. I noticed a vague Monday morning meeting when I logged on. Checked with my team to see if they knew what it was about and no one knew. Supervisor was MIA on slack. Just before it starts we got a group text from him that essentially said, “what the fuck. I’m so sorry guys. I’m not allowed to speak or I’m immediately fired”

    I checked the invite list and, sure enough… VP of department, VP of HR, my supervisor, and my small team. I instantly knew we were all fired.

    Joined the meeting a few minutes early and it was just my teammates all wondering out loud what’s going on. They’re all pretty young. Couldn’t help but blurt out, “nice knowing yall…”

    Supervisor texts me with “please don’t, we’ll grab a drink right after this”

    The cool executives log and blah blah blah your team is getting shuttered thanks bye.

    We did get drinks at 9:30 in the morning.

  •  Rev. Layle   ( @revlayle@lemm.ee ) 
    link
    fedilink
    English
    46
    edit-2
    10 months ago

    How about a meeting an hour after our daily standup and it’s the CTO and CIO saying “ok is everyone on the call?”

    I was telling myself “oh crap, the company is gonna shut down”

    5 minutes later “So as of this very moment, everyone stop working. The company is officially closed as of this meeting.”

    We were a start up essentially. Made it almost 9 years with ups and downs. This all happened at the end of July. At least got 3 month severance and insurance covered. I do start a new job Tuesday.

  • Sometimes you get the opposite too. Like an 8pm slack message stating the head of engineering has “decided to step down effective immediately” (aka: forced to resign). Which is a nice surprise cause you had a meeting booked with them for tomorrow.