Beehaw
  • Communities
  • Create Post
  • search
    Search
  • Login
  • Sign Up
Sören ( soeren@iusearchlinux.fyi )  to Programmer Humor@lemmy.ml · 3 years ago

scaling to millions

iusearchlinux.fyi

message-square
10
link
fedilink
275

scaling to millions

iusearchlinux.fyi

Sören ( soeren@iusearchlinux.fyi )  to Programmer Humor@lemmy.ml · 3 years ago
message-square
10
link
fedilink

https://www.levels.fyi/blog/scaling-to-millions-with-google-sheets.html

alert-triangle
You must log in or register to comment.
  • hobbsc ( hobbsc@lemmy.sdf.org ) 
    link
    fedilink
    arrow-up
    37
    ·
    3 years ago

    There’s a really fine line between needing a spreadsheet and needing a database and I’ve not yet found it. It’s probably more fuzzy than I realized but I have participated on so many programming projects that amounted to a spreadsheet that lived too long.

    • GenderNeutralBro ( GenderNeutralBro@lemmy.sdf.org ) 
      link
      fedilink
      English
      arrow-up
      41
      ·
      3 years ago

      Does it need to be accessed by multiple people? Does it need to be updated frequently? Does it need to be accessed programmatically? Does performance matter? If you answered “yes” to any of these questions, you should probably use a database.

      If it’s something you interact with manually, has less than 100,000 rows, and is mostly static, then sure, use a spreadsheet.

      I used to have some scripts to convert and merge between CSV and sqlite3. Even a lightweight db like sqlite3 has value. Google Sheets fills some of the gaps with its QUERY statement but I still find it a bit awkward to use a lot of the time.

      • Uprise42 ( Uprise42@artemis.camp ) 
        link
        fedilink
        arrow-up
        4
        ·
        3 years ago

        I can answer yes to all of these questions but still use a spreadsheet. I understand your point, but I feel even with these the line is still gray.

        I just checked and my largest spreadsheet currently has 14,300 lines across 12 tabs. Each tab contains the same information just pulled from a separate form. So each tab is linked to a form to update automatically when someone submits a new response. We then process these responses periodically throughout the day. Finished responses are color coded so a ton of formatting. Also 7+ people interacting with it daily.

        Then we have a data team that aggregates the information weekly through a script that sends them a .csv with the aggregate data.

        The spreadsheet (and subsequent forms) are updated twice a year. It was updated in June and will be updated again in December. It’s at 14k now and will continue to grow. We’ve been doing this through a few iterations and do not run into performance issues.

    • Big P ( peter@feddit.uk ) 
      link
      fedilink
      English
      arrow-up
      8
      ·
      3 years ago

      At some point you end up surpassing databases and end up with a giant pile of spreadsheets called a data warehouse

    • MonkderZweite ( MonkderZweite@feddit.ch ) 
      link
      fedilink
      arrow-up
      4
      ·
      3 years ago

      As soon as you stop data maintenance per hand, start using a db.

  • phorq ( phorq@lemmy.ml ) 
    link
    fedilink
    arrow-up
    34
    ·
    3 years ago

    The article doesn’t seem to say what type of database they moved to, I’d like to imagine it’s an excel spreadsheet…

    • mosiacmango ( mosiacmango@lemm.ee ) 
      link
      fedilink
      arrow-up
      1
      ·
      3 years ago

      deleted by creator

  • RagingHungryPanda ( RagingHungryPanda@lemm.ee ) 
    link
    fedilink
    arrow-up
    27
    ·
    edit-2
    2 years ago

    Removed by mod

    • Digestive_Biscuit ( Digestive_Biscuit@feddit.uk ) 
      link
      fedilink
      English
      arrow-up
      8
      ·
      3 years ago

      I can believe that. Where I work people use the Bi system to dump data out to Excel to then build the reports they want to see. Madness.

  • Awkwardparticle ( Awkwardparticle@artemis.camp ) 
    link
    fedilink
    arrow-up
    10
    ·
    3 years ago

    Not going to lie. I have made it really far using Google sheets as a database until I had to move to a full DB. Google Sheets is just an interface to a Google run DB anyways. A lot of the time I would just load sheets(tables) into data structures from a SQL library and use it like a regular database.

  • MonsiuerPatEBrown ( MonsiuerPatEBrown@reddthat.com ) deleted by creator
    link
    fedilink
    arrow-up
    4
    ·
    edit-2
    3 years ago

    deleted by creator

  • 👍Maximum Derek👍 ( Bishma@discuss.tchncs.de ) 
    link
    fedilink
    English
    arrow-up
    1
    ·
    3 years ago

    deleted by creator

  • Cwilliams ( Cwilliams@beehaw.org ) 
    link
    fedilink
    arrow-up
    1
    ·
    3 years ago

    https://www.youtube.com/watch?v=K6Vcfm7TA5U

  • katy ✨ ( cupcakezealot@lemmy.blahaj.zone ) 
    link
    fedilink
    arrow-up
    0
    ·
    3 years ago

    All the cool kids use Microsoft Access

Programmer Humor@lemmy.ml

programmerhumor@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programmerhumor@lemmy.ml

Post funny things about programming here! (Or just rant about your favourite programming language.)

Rules:

  • Posts must be relevant to programming, programmers, or computer science.
  • No NSFW content.
  • Jokes must be in good taste. No hate speech, bigotry, etc.
Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 1 user / day
  • 41 users / week
  • 42 users / month
  • 1.41K users / 6 months
  • 871 local subscribers
  • 43.7K subscribers
  • 1.25K Posts
  • 10.2K Comments
  • Modlog
  • mods:
  • AgreeableLandscape ( AgreeableLandscape@lemmy.ml ) deleted by creator
  • cat_programmer ( cat_programmer@lemmy.ml ) 
  • BE: 0.19.21
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code