Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "airflow"
-
Who was the shithead cumface dumbass idiot thinking that notebooks should have their cooling units at the bottom. How the fuck are they supposed to get air when sitting on my lap?10
-
To all the data engineers in here: WTF is going on in your field?
I've worked closely with a dozen data engineers in the last 5 years (and talked to friends and internet strangers about this and get similiar responses), mine if them seem to know how to use a computer!
They don't understand git, ORMs, best practices, how to use a terminal, DAGs (important for using modern ETL scheduling tools like airflow and prefext), etc
Guys with 10 years of experience on their resume and they can't wrap a model into a flask app with 1 endpoint. They'll reference local files on their machine in w jupyter notebook and are shocked it won't work on other computers!17 -
My GPU blocks the airflow from the lower front intake fan to the CPU, so I wanted to have a fan in the 5.25" drive bay directly targeting the CPU.
While that bay fits a 140mm fan nicely, there was no mounting point. I ended up making four fan struts out of the metal covers for the 5.25" inserts, the ones that you wiggle out. Drilled holes into the case, a bit of foam above and below the fan to seal the larger gaps, and done.
The trick is ofc that the 5.25" case covers are meshed and hence act both as air intake and dust filter. The CPU runs a few K cooler under load.
13 -
I bought a computer awhile back off Kijiji (Canada's craigslist) for a really good price. Today, decided I was going to upgrade the ram since I got a sick deal on some corsair vengence 8gb sticks online...
And just before installing it, I realize the fucker decided to use low profile RAM in his build for a reason: he (for some fucking reason) decided to route the airflow for the system by placing the cooling fan directly over the first 2 memory slots.
Guess who's 5 minute memory upgrade just turned into an hour of re-routing all the airflow in the PC and having to redo all the fan wiring.
I shouldn't complain, I mean I got this computer a couple years back for like $400, but still, wtf man...2 -
!rant
Rant from my previous work as a consultant Data Engineer (wish I had known this site back then).
During my stay at the place, we have a big client whose contact with us was an incompetent stressful fellow.
I single-handedly build a humongous automated data pipeline using Airflow. I am very proud of my baby as my first massive project and check it obsessively for every possible flaw, especially when writing down documentation for the poor soul that would take my place.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm. -
Airflow… airflow… I hate you so freaking much, you are a bloated piece of software that packages wayyy too much and increase the complexity of any solution we built on top of you. Plus unit-testing and integration tests are wayyy too difficult with you. But man… your recent UI changes are a massive welcome.
A year ago I was tasked with either upgrading airflow to version 2 or to migrate the code to another tool. Naively I thought “well might as well upgrade to avoid a rewrite”. Little did I know that the reason airflow didn’t scale out well for us, was due to people over the years not having a grasp on airflow primitives like “pools”, “workers” and “operators”. Ended up refactoring the entire codebase for both infra and DAGs anyhow AND upgrading the beast AND lower cost by a factor of 2 (from $100-$150 daily to $50-$70 daily).
But seriously feels like I could’ve solved the scheduling issue with literally any message queue+decent library (like celery or Faust) and I’d have half the headache.3 -
I started working with Luigi, the workflow manager for a data-science project. Any input on how to test the Tasks or how does it compare to Airflow?
-
Making distributed scheduler that queue and run tasks on containers or other executors in future and also pulls new tasks from defined git repositories.
Tasks are added based on simple yaml configuration.
Need that for my side projects that gather data from multiple sources from time to time.
k8s looks to heavy for that and airflow can’t be configured like I want it to be so I started writing my own on Monday.
Nearly finished poc version.2 -
I really want to switch my career from being a Full-Stack python/javascript developer to be a Data Engineer.
I've already worked with relational and non-relational databases, troubleshooted a couple of Airflow DAGs, deployed production-ready python code but now I feel kinda lost, every course I start on the Data engineering topic feels really useless since I feel like I've already worked with that technology/library, but I'm still afraid of start taking interviews.
Any good book/course or resource that I should look in?
BTW first rant in a couple of years, this brings me memories1 -
Working on a feature which heavies relies on a data pipeline. I noticed it is a couple of lambda functions calling each other ( Fuck you to the guy who made it). The best way to get sanity back is build a proper etl pipeline. Any suggestions for building a etl in python with reliability.
Options already considered
1. Celery tasks - Worked well but no overview of the single task progress across celery tasks
2. Airflow - Gives good overview but the docs make less sense than a 10 yr talking. Mostly because they introduced a new syntax and not everything has migrated fully yet. Also no support for reusing dags2 -
Does someone know how to set up plugins for airflow?
In particular do you know how I set the PYTHONPATH env var for the docker container to Import my custom airflow plugin? -
Window Replacement Columbus, Ohio: The Ultimate Guide
When it comes to enhancing the energy efficiency, aesthetics, and value of your home, window replacement in Columbus, Ohio
is a smart investment. Whether your windows are outdated, damaged, or simply not performing as they should, replacing them with high-quality, energy-efficient options can make a significant difference. This guide will walk you through everything you need to know about window replacement in Columbus, Ohio.
Why Replace Your Windows?
There are several reasons why homeowners in Columbus choose to replace their windows:
1. Improve Energy Efficiency
Ohio experiences a range of temperatures throughout the year. Energy-efficient windows help maintain indoor comfort by reducing heat loss in winter and keeping cool air inside during summer. Look for ENERGY STAR-rated windows for optimal efficiency.
2. Enhance Curb Appeal
New windows can drastically improve your home's exterior appearance. With various styles, materials, and finishes available, you can customize the look to match your aesthetic preferences.
3. Increase Home Value
According to industry reports, window replacement can offer a high return on investment (ROI). If you plan to sell your home in Columbus, new windows can attract buyers and increase your property's resale value.
4. Reduce Noise Pollution
Living in urban areas or near busy streets? Modern windows with double or triple-pane glass help reduce outside noise, making your home quieter and more peaceful.
5. Enhance Security & Safety
Older windows with weak locks and outdated materials can be a security risk. Newer models offer advanced locking mechanisms and shatter-resistant glass for added protection.
Types of Windows for Columbus Homes
When choosing replacement windows, consider the various styles available:
Double-Hung Windows – A popular choice with two sashes that move up and down, allowing for better ventilation.
Casement Windows – Hinged on one side and open outward, offering excellent airflow and an unobstructed view.
Sliding Windows – Move horizontally along a track, perfect for contemporary homes.
Bay & Bow Windows – Extend outward, adding space and natural light.
Picture Windows – Fixed and designed to provide a clear, expansive view.
Choosing the Right Window Material
Vinyl Windows – Affordable, durable, and low-maintenance.
Wood Windows – Classic and aesthetically pleasing but require regular upkeep.
Fiberglass Windows – Highly durable and energy-efficient.
Aluminum Windows – Strong and lightweight but less insulating than other materials.
Finding the Best Window Replacement Company in Columbus, Ohio
To ensure quality installation and customer satisfaction, consider the following when choosing a window replacement company:
Reputation & Reviews – Check online reviews and ratings on platforms like Google and Yelp.
Experience & Certifications – Look for companies with certifications from leading manufacturers or the National Fenestration Rating Council (NFRC).
Warranty & Guarantees – Choose a provider that offers warranties on both materials and labor.
Free Estimates – Get quotes from multiple companies to compare pricing and services.
Cost of Window Replacement in Columbus, Ohio
The cost of replacing windows varies based on factors such as:
Window size and style
Frame material
Glass type (double-pane, triple-pane, low-E coatings)
Installation complexity
On average, Columbus homeowners can expect to pay between $300 to $1,200 per window, including installation.
Conclusion
Investing in window replacement in Columbus, Ohio can significantly improve your home's comfort, efficiency, and value. By selecting high-quality windows and hiring a trusted contractor, you can enjoy long-term benefits and a great return on investment. Start your search today and find the perfect windows for your Columbus home!
6 -
Chimney & Stone Masonry LLC: Your Trusted Partner for Chimney, Fireplace, and Stone Services in Connecticut
At Chimney & Stone Masonry LLC, we pride ourselves on being Connecticut’s premier provider of comprehensive chimney, fireplace, and stone repair services. With years of experience serving homeowners and businesses in New Britain, CT, and throughout the region, our team is dedicated to offering top-tier quality in every aspect of our work. Whether you need regular chimney cleaning, fireplace inspections, or complete masonry installations, we are here to ensure that your property remains safe, functional, and aesthetically pleasing.
Comprehensive Chimney Services
A chimney is an essential part of your home, especially in Connecticut’s colder months, when a warm fire becomes a central part of family gatherings and daily comfort. However, like all systems, chimneys require regular maintenance to perform optimally and safely. At Chimney & Stone Masonry LLC, we offer a broad spectrum of chimney services to meet your needs:
Chimney Cleaning: Over time, chimneys can accumulate soot, creosote, and debris, creating a fire hazard and blocking the venting of harmful gases. Our chimney cleaning services are designed to remove all harmful buildup, ensuring a clean, safe, and efficient chimney system. Regular cleaning not only enhances safety but also prolongs the life of your chimney.
Chimney Inspection: Every chimney needs periodic inspections to ensure it is in good working condition. Whether you're preparing for a season of use or simply want peace of mind, our certified professionals perform thorough chimney inspections. We check for signs of damage, such as cracks, leaks, or other issues that could jeopardize your home’s safety.
Chimney Repairs: If your chimney has sustained damage from weather, age, or lack of maintenance, our skilled team can address a variety of chimney repair needs. From fixing masonry issues like cracked bricks and deteriorating mortar to repairing chimney caps and liners, we are equipped to restore your chimney to full functionality.
Chimney Installations: If you are looking to add a new chimney or replace an old one, Chimney & Stone Masonry LLC can handle your installation from start to finish. We use high-quality materials and adhere to the best industry practices to ensure your new chimney performs efficiently and safely for years to come.
Fireplace Services to Enhance Your Home
The fireplace is the heart of many homes, providing warmth, comfort, and a cozy atmosphere. Whether you are looking to restore an old fireplace or install a new one, Chimney & Stone Masonry LLC has the expertise to make your fireplace a focal point of your living space. We offer:
Fireplace Repair: Over time, fireplaces can experience wear and tear from constant use. Our team can fix issues like cracked fireboxes, damaged hearths, or malfunctioning dampers, ensuring your fireplace works safely and efficiently.
Fireplace Installation: Whether you're building a new home or remodeling an existing space, our team specializes in custom fireplace installations. We can design and install fireplaces that match your aesthetic preferences while meeting all safety standards.
Fireplace Cleaning: Just like chimneys, fireplaces require regular cleaning to maintain their performance. Our team will thoroughly clean the firebox, hearth, and chimney to ensure optimal airflow and a safe burning experience.
Stone Masonry Services
In addition to our chimney and fireplace services, Chimney & Stone Masonry LLC is also a trusted provider of stone masonry services. Our skilled masons can enhance the beauty and functionality of your home with custom stonework. Whether you’re adding a stone patio, retaining wall, or fireplace surround, we use high-quality materials and craftsmanship to bring your vision to life.
Stone Repairs: Stone structures, such as walls, chimneys, and walkways, can experience damage due to weather, shifting foundations, or natural wear. Our team offers stone repair services to restore the beauty and functionality of these features.
Stone Installation: Whether you're building a new feature or renovating an existing one, our stone installation services ensure that your project is completed with precision and attention to detail. We can help you choose the best type of stone to suit your style and budget.
Custom Masonry Projects: At Chimney & Stone Masonry LLC, we work with you to design custom stone features that enhance the aesthetic appeal of your home or business. From custom stone fireplaces to decorative stone walkways, we can create unique elements that reflect your personal style1 -
When it comes to healthcare professionals, uniforms are not just about appearance—they play a key role in daily performance. For men working in hospitals, clinics, or medical facilities, the right scrubs for men can significantly affect comfort, focus, and overall work performance. Whether you are a doctor, nurse, or any other medical professional, understanding the factors that impact your choice of male scrubs can make a difference in how efficiently you work.
visit: lintex.in/category/male/
Durability
Working in a healthcare setting often exposes your uniform to rough conditions—spills, stains, and wear and tear. The best mens scrub options are made from durable fabrics that can withstand daily use and frequent washing. Scrubs made from lower-quality materials can fade, shrink, or lose their shape quickly, resulting in the need to replace them more often.
Durable male scrubs save time and money in the long run, ensuring you don’t have to keep buying new ones. This also adds peace of mind, as you won’t be distracted by the state of your uniform and can stay focused on patient care. When buying scrubs for men, it’s essential to consider durability to maintain a professional appearance throughout your shifts.
Breathability and Temperature Control
Healthcare professionals spend long hours on their feet, often in warm or stuffy environments. Wearing mens scrub sets made from breathable fabrics is essential to stay cool and comfortable. Scrubs with moisture-wicking properties are designed to draw sweat away from the body, keeping you dry and preventing body odor.
The best medical scrubs for men are made with fabrics that allow for airflow, preventing overheating and promoting comfort in both high-pressure and high-temperature environments. When your body temperature is regulated, it’s easier to stay focused and perform tasks without feeling distracted by discomfort.
Comfort and Fit
One of the most important factors affecting work performance is the fit and comfort of mens scrubs. Scrubs that are too tight or too loose can restrict movement, making it difficult to perform essential tasks. Best male scrubs are designed with a comfortable fit that allows healthcare professionals to move freely, bend, stretch, and lift without any discomfort. Scrubs that fit well reduce distractions, allowing you to focus on your patients and work at your best.
Additionally, the fabric’s softness is crucial for comfort, especially during long shifts. Best medical scrubs for men are often made of materials like cotton blends or moisture-wicking fabrics that provide comfort all day long. If your scrubs are uncomfortable, it could lead to irritation or fatigue, which can hinder your performance.
Pockets and Practicality
As a healthcare professional, you need quick access to medical tools and instruments like pens, thermometers, and stethoscopes. The placement and number of pockets in male scrubs can play a big role in your ability to keep essential items within reach. Scrubs with multiple, well-placed pockets allow you to organize your tools and supplies efficiently.
Choosing the best male scrubs that offer functional pockets can boost your productivity. Scrubs with extra pockets on the chest or thighs give you the ability to carry all your tools without needing to leave your station or interrupt your workflow. This feature directly impacts your ability to stay organized, focused, and on-task throughout the day.
Flexibility and Range of Motion
Another important factor that affects work performance is the level of flexibility offered by your mens scrubs. Healthcare professionals need scrubs that allow for a full range of motion to perform physical tasks such as lifting, bending, and assisting patients. Scrubs that are too stiff or tight can limit your movements, making it harder to perform hands-on tasks efficiently.
When you choose the best medical scrubs for men, opt for scrubs with stretchable fabric blends that provide flexibility without compromising comfort. This ensures that you can move easily and focus on your work without feeling restricted by your uniform.
Professional Appearance
The way you look in your scrubs plays a role in your confidence and professionalism. Scrubs that are well-fitted and stylish contribute to a more polished appearance, which can influence how patients and colleagues perceive you. Wearing the best mens scrub that fits your personal style while adhering to professional standards can make you feel more confident and prepared.
When you look professional, it not only boosts your self-esteem but also helps build trust with your patients. A neat, tidy, and well-maintained uniform is a sign of competence and care, which can positively influence your interactions with others in the healthcare setting.
1
