Beehaw
  • Communities
  • Create Post
  • search
    Search
  • Login
  • Sign Up
Raphael ( rglullis@communick.news )  to Programming@programming.devEnglish · 3 years ago

One line of code caused AT&T to lose $60 million

engineercodex.substack.com

external-link
message-square
13
link
fedilink
88
external-link

One line of code caused AT&T to lose $60 million

engineercodex.substack.com

Raphael ( rglullis@communick.news )  to Programming@programming.devEnglish · 3 years ago
message-square
13
link
fedilink
How one line of code caused a $60 million loss
engineercodex.substack.com
external-link
60,000 people lost full phone service, half of AT&T's network was down, and 500 airline flights were delayed
alert-triangle
You must log in or # to comment.
  • drolex ( drolex@sopuli.xyz ) 
    link
    fedilink
    arrow-up
    49
    ·
    3 years ago

    If one single line of code can make you lose $60M, surely you’ll ensure due review processes and independent QA and clear requirements and regular audits and a middle management not only doing KPI monitoring for a failing upper management. Right? Rrrright?

    • BeardedGingerWonder ( BeardedGingerWonder@feddit.uk ) 
      link
      fedilink
      English
      arrow-up
      8
      ·
      3 years ago

      Hahahaha, you’re kidding right? I shit you not, I’ve literally seen a single line change almost cost a company £150MM during testing because “we need to test in prod because the guy we need to run the test hasn’t got access to the QA environment”

      Best part was the actual change, there was a bug where a number that should’ve been divided by 100 was being multiplied by 100, the dev somehow managed to implement the fix in such a way that the number was multiplied by a further 100.

      • snowe ( snowe@programming.dev ) 
        link
        fedilink
        arrow-up
        3
        ·
        3 years ago

        Knight Capital Group lost $440 million in just 45 minutes due to a repurposed feature flag.

        https://www.henricodolfing.com/2019/06/project-failure-case-study-knight-capital.html?m=1

        Kinda makes the att one seem tiny in comparison.

  • beefpeach ( beefpeach@infosec.pub ) 
    link
    fedilink
    arrow-up
    33
    ·
    3 years ago

    Recently, I paid ATT $500 to close my account and zero it out.

    Fast forward 6 months, they send me a letter saying they need $300 more for cancellation fees or it will be sent to collections.

    Write all the bad code you can, papi.

    • heeplr ( heeplr@feddit.de ) 
      link
      fedilink
      arrow-up
      3
      ·
      3 years ago

      Sure it’s cancellation fees? This doesn’t seem legal.

  • boredtortoise ( boredtortoise@lemm.ee ) 
    link
    fedilink
    arrow-up
    22
    ·
    3 years ago

    Someone should write more of those lines

    • Raphael ( rglullis@communick.news ) OP
      link
      fedilink
      English
      arrow-up
      3
      ·
      3 years ago

      So you want more people to lose full phone service?

      • 𝒍𝒆𝒎𝒂𝒏𝒏 ( lemann@lemmy.one ) 
        link
        fedilink
        arrow-up
        4
        ·
        3 years ago

        Would get more people upset that they don’t have an alternative (assuming this affects one of those cable monopoly communities)

      • drolex ( drolex@sopuli.xyz ) 
        link
        fedilink
        arrow-up
        3
        ·
        3 years ago

        I work for the Atlantic and Pacific Telegraph Company, and I think I talk for everyone when I say: yes, why not

  • spudwart ( spudwart@spudwart.com ) 
    link
    fedilink
    English
    arrow-up
    11
    ·
    3 years ago

    Damn, if only the systems your phone network were ran on hadn’t been forcibly closed source and scared good devs from interacting with you because of your sue-happy nature regarding BSD/Unix.

  • 0x0 ( 0x0@programming.dev ) 
    link
    fedilink
    arrow-up
    3
    ·
    3 years ago

    And it wasn’t COBOL.

  • Neato ( Neato@kbin.social ) 
    link
    fedilink
    arrow-up
    2
    ·
    3 years ago

    We’ve had one, yes.

  • darkfiremp3 ( darkfiremp3@beehaw.org ) 
    link
    fedilink
    arrow-up
    2
    ·
    3 years ago

    I feel like anyone doing any automation with aws could hit this

Programming@programming.dev

programming@programming.dev

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programming@programming.dev

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person’s post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you’re posting long videos try to add in some form of tldr for those who don’t want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 134 users / day
  • 549 users / week
  • 1.75K users / month
  • 4.16K users / 6 months
  • 449 local subscribers
  • 28.7K subscribers
  • 2.94K Posts
  • 23K Comments
  • Modlog
  • mods:
  • snowe ( snowe@programming.dev ) 
  • Ategon ( Ategon@programming.dev ) 
  • UlrikHD ( UlrikHD@programming.dev ) 
  • bugsmith ( bugsmith@programming.dev ) 
  • Spyro ( Spyro@programming.dev ) 
  • BE: 0.19.15
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code