Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "aws configuration"
-
I'm a DevOps engineer. It's my job to understand why this type of shit is broken, and when I finally figure it out, I get so mad at bullish players like AWS.
It's simple. Install Python3 from apt.
`apt-get update && apt-get install -y python3-dev`
I've done this thousands of times, and it just works.
Docker? Yup.
AWS AMI? Yup.
Automation? Nope.
WTF? Let's waste 2.5 hours and figure out why this morning.
In docker: `apt-cache policy python3-dev` shows us:
python3-dev:
http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
But in AWS instance, we see we're reading from "http://us-east-1.ec2.archive.ubuntu.com/... focal/main" instead!
Ah, but why does it fail? AWS is just using a mirror, right? Not quite.
When the automation script is running, it's beating AWS to the apt mirror update! My instance, running on AWS is trying to access the same archive.ubuntu.com that the Docker container tried to use. "python3-dev" was not a candidate for installation! WTF Amazon? Shouldn't that just work, even if I'm not using your mirror?
So I try again, and again, and again. It works, on average, 1 out of every 5 times. I'm assuming this means we're seeing some strange shit configuration between EC2 racks where some are configured to redirect archive.ubuntu.com to the ec2 mirror, and others are configured to block. I haven't dug this far into the issue yet, because by the time I can SSH into the machine after automation, the apt list has already received it's blessed update from EC2.
Now I have to build a graceful delay into my automation while I wait for AWS to mangle, I mean "fix up" my apt sources list to their whim.
After completely blowing my allotted time on this task, I just shipped a "sleep" statement in my code. I feel so dirty. I'm going to go brew some more coffee to be okay with my life. Then figure out a proper wait statement.7 -
Despite common sense, I think technology is not making our lives easier. It's just build chaos on top of chaos.
Take server-side programming for instance.
First you have to find someone to host your thing, or a PaaS provider. Then you have to figure out how much RAM and storage you need, which OS you're going to use. And then there's Docker (which will run on top of a VM on AWS or GCP anyway, making even less sense). And then there's the server technology: nginx, Apache (and many many more; if, that is, you're using a server at all). And then there are firewalls, proxies, SSL. And then you go back to the start, because you have to check if your hosting provider will support the OS or Docker or your server. (I smell infinite recursion here.)
Each of these moving parts come with their own can of worms in terms of configuration and security. A whole bible to read if you want to have the slightest clue about what you're doing.
And then there's the programming language to use and its accompanying frameworks. Can they replace the server technology? Should you? Will they conflict with each other and open yet another backdoor into your system? Is it supported by your hosting provider? (Did I mention an infinite recursion somewhere?)
And then there's the database. Does it have a port to the language/framework of your choosing? Why does it expose an web interface? Is it supposed to replace your server? And why are its security features optional again? (Just so I have to test both the insecure and the secure environments?)
And you haven't written a single line of code yet, mind you.4 -
When I saw a blog post said you need more than 1 cup of coffee to get AWS configuration done, I didn't believe it. Now I know... it's true. very true. It takes forever to make use of your config. By the time it's done, you already switch to something else.
-
What is the smallest, most innocuous mistake you've made that had huge consequences later?
I'll start: today I made a one-letter typo in a configuration file, which set off a ridiculous comedy of errors that culminated with me tearing down and rebuilding a whole AWS account.1 -
AWS Contractor
I've been putting a web application together that I'm looking to have published on AWS. Not having too much experience with AWS, I am looking to hire a contractor. I've had a number of quotes from different AWS admin's ranging from $40 an hour to $200 an hour, from 1-days worth of work to 2-months worth of work!
I'm not really sure what to make of it or to whom to trust. I believe they’re using my ignorance to overcharge me. I've listed my requirements below, could you guys use your professional experiences to let me know what you think is reasonable charge and where best I could find someone to help me.
My application is a US shopping website where people can set up an online shop and upload their products and maintain an inventory of the items.
This is what I’m looking for setup and configuration with the following two areas:
1) AWS SYSTEMS…
* AIM - Set up my server admin users.
* EC2 - Web Hosting.
* RDS - Fast DB.
* SES - To send emails.
* S3 Buckets - Uploaded image hosting.
Route 53 - I don’t know but someone said I should have this.
* Elastic Load Balancing - For, well, load balancing.
2) SCRIPTS…
* A script that would back up the database once a day and save it to a private S3 Bucket.
* A script that will run once a day that calls an internal API, and POST a query to it.
* A script that runs once every 90 days, to refresh the SSL using ZeroSSL.com
Is there anything that I've missed such as security systems, firewalls, auto scaling and CDNs?
The quotes that I've received arranged from $320 to $64,000. I know I am being abused because of my ignorance. I would never overcharge someone because the customer doesn't know the efforts of the work. I hope someone here can help to understand the efforts needed and can tell me the true cost.
Thank you6 -
Let's just say that I hate configuration of any kind.
AWS IAM, Cloudfront... it all makes me want to end it all.
Why can't this stuff just work?4 -
New to AWS, is my best option for having a integer value (maximum number of items to process) that I can override for a lambda step function
to read a value from an S3 bucket where I can overwrite the value if I want to change it. This seems silly and I feel silly as I expect my situation should be simple and not novel at all.
For some reason I expected I could use an environment variable, but didn't see an option to overwrite it in the web GUI https://docs.aws.amazon.com/lambda/...2 -
Hi,
So I have been using colab for the past 2 years. I liked how without any setup you can use kernels with GPU and TPU with some configuration.
But recently I can't train any model. It always goes runtime error, runtime disconnected, not to mention they have limited their total hours of usage for a day.
I know you are providing everything for free but this is just annoying. I dont mind if google wants to start a subscription plan for colab...its much better for fast prototyping than getting a cloud server from google or aws or anything of such sorts.
I have been trying to train a model with only 3 gigs of data and I cant complete the model, once I change the tab it shows Runtime Disconnected. DAMN it.
Sadly, I am trying not to use colab from now on.
But yeah I am frustrated with colab and their services.3 -
Brilliant rant from Redditor OK6502 in a thread about a "tech screen" being used to get free labor:
Usually when something like this uses the words complex tech stack it means you're going to have to deal with shitty server code distributed over a mix of Azure and AWS nodes and a lone Linux server running under someone's desk, an infuriating configuration hell with no safeguards for keeping dev and prod isolated, a hodge podge of different scripting languages (why not make scripts in pero that call power shell which then calls more perl? Should work right?) and random but critical shit checked into 3 different SVN, stuff stashed on people's shares that will never be checked even though you can't do your homework b without it, usually copied from someone else's share who left the company 3 years ago, no QA process to speak of (while claiming to be agile, somehow) and a front end that is maintained by one exhausted junior dev who inherited a mess of 20 different js frameworks that all load at the same time with every single click, somehow.
The full thread is really worth reading:
https://reddit.com/r/... -
Hey errrbody!!!
I'm banging out a couple "showcase" mobile apps for practice, portfolio, and/or as potential templating tools.
I have no issue writing the code, I just wanted to see if I could get a couple pointers as far as user databases go. I'd like to have some "user profile" features generated from a FB...vlike profile images, name, address, contact, yadda yadda yadda. I usually use Firebase, but I am still having a little trouble with the more advanced stuff when it comes to integrating users profile data. I can get values from Google and whatnot, but I'd like to see what my other options are on the smaller scale.
I am currently writing code in Flutter/Dart, ReactJS( not native!), Vanilla Js, Python, and CPP.
I know there's options for client side storage like Shared Prefs, Sqflite, etc, as well as server/DB side stuff like Firebase, Aws, Mongo, Node, SQL, etc- you get the idea.
I just want something with decent documentation that's reliable, not a massive undertaking (at least not for all this little stuff, anyways) and could potentially be a go-to platform configuration in the future. It'd be cool to wire in my Flutter and js shit of possible, bit honestly I'm cool with having separate setups for the time being. Any extra input regarding the use of python and/or cpp as well (either separately or with mobile) would be rad as fuck!!!
I do realize it's a pretty vast area to cover, but I figured it couldn't hurt to see what everyone likes to use for full-stack setups.
Thanks!!!!9