Grid Status | Senior Software Engineer | Remote or Hybrid Chicago | Full-time | https://www.gridstatus.io/ | $150k+ salary & equity
Grid Status is one of the most widely used energy data platforms in the world. Our data has been used in reporting and analysis by the Wall Street Journal, New York Times, the Federal Reserve, and by thousands of users who regularly visit our products every month.
As one of the first members of our engineering team, you will have a hand in both defining and implementing the product we deliver to customers. No experience in the energy industry is necessary—just a strong eagerness to learn. Our team’s passion and expertise will help you quickly get up to speed.
Our stack is mostly: python, react, postgres, snowflake, AWS
Those links are very informative. Can you elaborate on Ancillary Services Monitor and Energy Storage Resource Action dashboards? What’s the installed capacity that supported?
Also your original post link states event happened at 7:02 AM, your links here points to 8:05 AM. Can you explain this?
Just to clarify and understand what happened, I believe right after the trip some generators all around the grid picked up the load (unless UFLS was activated) immediately (around 7:03), we can call these generators support system. Then around 7:05 batteries kicked in with 468 MW, as a support to support system.
Grid Status | Senior Software Engineer | Remote or Hybrid Chicago | Full-time | https://www.gridstatus.io/
Grid Status is one of the most widely used energy data platforms in the world. Our data has been used in reporting and analysis by the Wall Street Journal, New York Times, the Federal Reserve, and by thousands of users who regularly visit our products every month.
As one of the first members of our engineering team, you will have a hand in both defining and implementing the product we deliver to customers. No experience in the energy industry is necessary—just a strong eagerness to learn. Our team’s passion and expertise will help you quickly get up to speed.
Our stack is mostly: python, react, postgres, snowflake, AWS
The US electrical grid is changing fast. I built a site to track instantaneous solar generation and other records in real time across different parts of the country.
New solar records are done until next summer but still interesting stuff happening. For example, California hit a new battery charging record a few weeks ago.
There is lots of potential, but as we see in Germany, a glass ceiling for solar and wind power exists where backup is needed - when there is little wind for only one hour at night, you still need another controllable source of energy with an output that does not vary by weather. Few are available with low CO2, such as hydro, nuclear, geothermal
It’s a wide range of sources but basically the different Independent System Operators in the US and the Energy Information Agency (EIA) All of our data scrappers are open source: https://github.com/kmax12/gridstatus
I am one of the people who built this page and the Grid Status site. Cool to see this posted!
Ultimately, the solar eclipse this weekend came and went without any noticeable impact for consumers of electricity. In my opinion, this is a very notable feat.
A well-planned and executed response by grid operators to a predictable eclipse bodes well for a future where events like a particularly cloudy day, blizzard conditions across a large area, or widespread hail damage to solar farms could reduce generation by similar MW values in the middle of the day.
As the grid becomes a more dynamic and volatile place, making operational and investment decisions based on data is more important than ever before.
If you're curious about this data for the United States, I built a site that lets you track renewable production, as well as many other interesting datasets about the US electrical grid.
We focus on the United States, so we can have the deepest coverage for each of the regions. In some cases that means were the same as Electricity Maps. In other case, you'll see our data is more real time without relying on estimations.
We have a lot more than generation mix data. For the wholesale energy markets, we also have all the pricing data[1]!
I notice that the map uses OpenStreetMap (via Mapbox, it looks like) for its base data, but doesn't display the required attribution[0]. For fixing this, their Attribution Guidelines[1] are pretty informative. Mapbox also has some helpful docs[2], and may have some additional requirements. Thanks!
Edit: After a bit of digging, I'm a bit unimpressed: it looks like the OSM and Mapbox attributions are deliberately hidden? From your compiled index.css:
That's awesome!
Why is the PNW power-source display an x-y plot and the Midwest and Texas and New England (the others I looked at) are pies?
I see in the PNW plot there's a ~constant output from nuclear and the other sources are more variable (x-t plot makes it easier to see this) but nuclear is a source of energy in the other locations with pie charts, as well.
Nuclear is strictly a base load power source. It takes on the scale of "many hours" to days to change generation level at a nuke, and there's intense economic pressure to operate them at full capacity as much as possible because its opex is largely fixed and enormous (as is the capex from construction.)
This is why all the people shrieking about how evil the public was/is for 'irrationally' hating nuclear and how we need more nuclear (here, on twitter, and reddit) are idiots, especially when they talk about it providing "grid stability" because of those naughty, highly variable renewables.
Grid stability comes from fast-reacting power sources like natural gas, hydro, and energy storage systems like pumped hydro and batteries. The most famous example of this is in the UK, where pumped hydro is fast enough to react to "The Great British Kettle Surge" aka TV pickup:
Any time someone opens their mouth and says nuclear is the solution to "days with no sun or wind", you can immediately discount everything else that comes out of their mouth as being from a person who doesn't understand the first thing about electrical grids/generation.
Even if it were feasible to change power levels fast enough to react to wind and solar generation changing from a physics standpoint, no power authority in the world would keep nuclear generation (the most expensive form of power generation already) capacity in reserve for a cloudy day.
What we need are more renewable sources, more energy storage, and a wide-scale HVDC distribution network to be able to cost-effectively move power from areas where there is strong generation by renewables to areas that need it, or at least into storage.
It depends on the source of the data. For the regions with the pie charts, those are Independent System Operators (ISOs) or Regional Transmission Organizations (RTOs). They publish much more data and is more real time than the other regions, so we have different pages to display them.
If this was constructive criticism I wouldn’t have said anything. Not that hard to clean it up, if you want to use an article about Portugal’s renewable policy to make a point about homelessness in California, and the governance of it, then just make your point without demonizing people, give constructive criticism (i.e. this is bad because the outcomes for society are bad, as evidenced by [x] and [y] and I think California should be doing Z, which worked in Z country, here are the outcomes: [z_link]
Cool. Take it somewhere else. You can pop off on X or Facebook or whatever, that content doesn’t belong on HN.
You’ll get flagged and removed on HN for talking like that. Check the contribution guidelines, ask @dang yourself if you don’t believe me.
I’m sure you’re chaffing at me even suggesting moderate speech here, but this corner of the internet isn’t polluted like the others. I don’t use social media, I use this, for that reason. I’m just asking you to abide by the ground rules
Much respect for the amount of ETL you must have written to accomplish this. I have a local dashboard that duplicates some, not all, of your features for CAISO only and that was more than enough work.
ETA: Can I ask why you don't have records for CAISO storage charge level?
I’m curious why there appears to be a pretty significant delta between their data and yours.
Also, if you’re open to suggestions, I had trouble finding the pricing for gridstatus. Entirely possible I was missing something obvious, but I wanted to see how much it might cost to get the long term Ercot generation-by-source dataset and couldn’t seem to get a clear answer.
Regardless, building things like this takes a ton of effort, and I appreciate all you’re doing. Keep up the great work!
we report the instantaneous load records while that page is reporting the hourly average. I wouldn’t say the results are that different. They point to about the same times even if numbers have a small delta.
My focus right now is to make this data more available and easier to use. Haven’t done much to monetize yet. If you just want that data source, happy to get it to you for free. Shoot me an email at max@gridstatus.io or sign up for a free api key and grab it from the site
The technical term for long-distance power delivery is "transmission". Distribution refers to getting the power from the local substation to the individual homes and businesses.
Now that you pointed it, the patent-hell around molten sodium batteries should have ended already. Yet, I haven't hear about anybody doing anything with them. Is there some action happening?
If you think gas is only 50% more polluting than making a solar panel, yeah.
I don't think that. Some gas and oil companies have, via their lobbies, produced content that implied this is true with articles that complain about the various processes over the lifetime of a solar panel that produce waste, but I think even they have refrained from making the claim that it's almost as bad as gas.
Easy technical remedy -- add in storage on key nodes - sell it back to the grid in the evening or when its not high or bid back in on capacity markets.
Challenge is getting the pricing model correct for such ancillary grid benefits to make it worth it for developer to build.
"Just do storage"... It's one of the hardest problems in the entire energy grid and on of the real sticking points of renewable energy, it's quite hard to store appreciable amounts of energy.
Pumped hydro is one of the better options but it has the ecological impact of just building new hydro power but less great economic impacts because the water level is less stable for people to live beside.
The numbers really aren’t that insane when you consider how quickly battery production has been ramping up. Nearly 100% of passenger cars going EV fairly quickly isn’t crazy looking at adoption curves. It’s likely those slow down soon, but continuing to build factories just as fast for grid storage is perfectly reasonable.
There’s over 280 million cars in the US, assuming on average that’s ~75kWh each we’re looking at ~21 tWh worth of battery storage. Meanwhile the average daily electricity use in the US is currently only 11 tWh. Of course that increases in a 100% EV world but EV’s are generally quite flexible demand.
PS: Solar power plants are often built to store ~50% of their daily output in batteries. It’s currently economically viable because that’s released at peak demand and thus peak prices, but with how quickly battery prices have been falling they will soon be viable even for normal nighttime prices.
Back feeding and metering all that power isn't exactly simple and it's also a big economic cost to just shove onto people in the form of increased battery wear and effectively reduced range since the most common trips begin early in the morning before renewables like solar come back online to produce the excess needed to recharge all the cars you've borrowed power from over night. Because of that cost that kind of distributed battery power is going to come out quite expensive unless you're just vastly underpaying for the depreciation of the battery like Uber does for it's drivers' cars wear and tear.
Another big thing is your 21 TWh number has huge flaws. That's fully draining everyone's batteries to provide the power and no one is going to accept that we want the cars because we need to go places. Next we're decades away from even getting close to full EV penetration. Even if we stop selling new ICE vehicles they're durable goods that people hold on to for a long time. It's the kind of 'Company/technology X takes over everything' assumption that so vastly inflates the valuations of tech companies all over the sector.
It was simply an order of magnitude comparison for manufacturing capacity. Slowing EV adoption at say 50% penetration would be beneficial for the grid as that excess capacity in mining and manufacturing could be used for grid batteries.
I doubt the the cheapest grid has anywhere close to 21 tWh as having more generation is more useful. Hydro is extremely flexible and reliable can fill in for a modest and predictable multi day shortfall. On the other hand if a power plant is down then batteries get extremely expensive.
That said, the cost of batteries isn’t simply a function of total capacity as unused capacity extends battery lifespan. The actual grid will take this into account which then impacts spot prices etc. Nobody operating grid scale batteries is going to 0% for wholesale prices of 5c/kWh but you bet your ass they’re going to 0% for 1$/kWh.
How many years do you have to go back before "just roll out lots of solar" was the silly hippy-dippy answer that all the sophisticated commenters who got their information via unofficial fossil fuel PR laughed at?
Remember when solar was the big problem that no one had a solution for? Turns out we did.
The future of energy is a lot of solar and a lot of batteries. Some other stuff will be involved but those two will do lots.
We're already at the point where new build pumped hydro doesn't make financial sense unless you have other needs for a big pile of water. Solar and batteries will beat it.
And pumped hydro is limited by geography. Unless the local landscape supports one lake above the other (for some suitable definition of "above") you can't build pumped hydro at any significant scale.
You can build them all over the mountains but it involves flooding new valleys with the associated cost both economic from displacing the existing people and destroying the ecosystem in that valley.
Interesting! But 50kWh seems pretty small for (macro-) grid applications. How big are these units? Seems like most grid installations would want on the order of hundreds (~10MWh) to a few thousand (~100MWh) of these.
The focus is on microgrids, but they're modular and can be installed below grade so you can add as many as needed. Approximately 2m diameter by 1m tall.
We're still working out costing, but it might even make sense for residential use to take advantage of time-of-use rates or energy arbitrage. Other applications are industrial processes that require high power for short periods.
Cool! I'll definitely be following your progress with interest :)
Hopefully constructive feedback: I would put that residential thing further down on your list. I think that stuff tends to be too complicated unless / until you can aggregate a lot of loads (eg. I think Tesla only recently recently started doing this stuff residentially. Probably awhile before you reach their scale...)
I hope flywheel storage takes off (no pun intended) it seems such a logic solution when compared with water elevation or chemical solutions. Why hasn't it gone mainstream yet?
I remember being really hyped about flywheel energy storage... 20 years ago. I wonder if it has become more viable since then? And if so, what changed to improve viability?
In the past they have been made to work, but they were not much better than batteries. With the price of batteries dropping like a stone I suspect they are going to have an even harder time competing.
There are several differences from batteries. Generally the benefits are faster charge and discharge rates, unlimited cycles without degradation, no lithium or rare earths needed, and wider operating temperatures.
They are not good at long term storage though, as the self discharge rate is high. And historically they have been expensive.
They have become more viable thanks to higher efficiency motors/power components and magnetic bearings, among other things. The biggest drawback is still cost, which is what we're tackling.
(on a smaller scale, to date, Tesla has deployed Powerwalls and Powerpacks at more than 50,000 sites worldwide; their Lathrop, CA facility is ramping to manufacture 40GWh/yr of capacity)
Two things:
(1) Price signals have to be further clarified especially at a utility level for ancillary services
(2) Battery system prices have been dropping quickly mainly as a function of Chinese manufacturers building out quickly e.g. CATL. Tesla is also helping.
Competition is already there - its next how do you deploy your development costs for winning those assets.
Dual feed power line, one for home electrics, one for immersion heater. Store excess renewably generated energy in household hot water tanks during the day, to be used in the evening in place of fossil-fuel generated supply, which also evens out load peaks as a side effect. This also works well with home solar.
I think grid load management for water heaters is already fairly common. I've basic versions of the concept in use by a coop in South Carolina, and I can't imagine them being anywhere near the bleeding edge.
The Seebeck effect will definitely have less than 8% efficiency with 80C water. Perhaps GP means that the hot water will already be available to use for hygiene and laundry, which for those with an electric water heater, is a large portion of the household power draw.
I'm not sure if a household sized water tank-full could provide heat over the course of a cold night, and whether a heat exchanger for air heating or water pipes would be more efficient. I suppose it depends on the insulation and placement of the ducts and pipes and how much of the heat makes it to and through a wall.
I did some very crude calculations but assuming 50 gallons at 60C and 1000W energy loss per hour from a moderately insulated house on a 50F night, the full water heater could keep the house at 70F for 5.14 hours. Someone with more recent practical physics usage is welcome to check this figure.
Not at all. Peak shaving with approaches like this is fairly common with utilities. Lots of them will give huge rebates on smart meters in exchange for this.
The energy difference between ambient-temp-to-frozen and ambient-temp to steam is much larger. I would think that this affects scalability.
I live in a city center that uses chilled water for some use cases, but it certainly does not seem scalable enough to be an "easy technical remedy" to the issues of distribution being expensive.
Not quite but also, you know power trading markets have existed for a long time right and the lights stay on - probably a 99.9% uptime ;)
edit: Just re-read your comment I assumed you were implying high frequency trading as a bad thing - though it might have been to help non-energy people understand.
Well, you're not wrong! I understand power arbitrage is already a thing and for the most part it's OK. But once in a while you get situations like the crazy contracts in Texas a few years back. And I am a little concerned that if we create a large enough storage market controlled by a few big enough players, they could play financial games with a critical commodity, ie hoarding both generation and storage for peak resale value.
In both situations maybe there's an argument for market efficiencies and liquidity and such. But it scares me a little.
I am not a financial person (in the solar field, but not markets). I could totally be talking outta my ass!
There's always a risk of bad actors in any market. Hopefully we have learned and not forgotten the california energy crisis and Enron from the early 2000s. There are regulators who work on this though the market bad actors normally get caught a couple years after the fact. As long as the regulators keep catching them it seems likely that we won't have as many cheats.
The energy markets are significantly layered in power contracts that I would think it would be difficult for an energy storage provider to play that much of a position.
Well, there was this one energy trading company called Enron, and the power outages in California engineered to boost their profits. I'd say that some amount of weariness is justified in this case.
Why storage rather than using when available, eg running fridges/freezers/Aircon slightly cooler, water heaters hotter, charging cars, running the washing machine / dryer.
Industrially smelting aluminium, making hydrogen, heating water, desalination.
Not that I'm saying storage shouldn't be part of the mix.
Over longer distances it is more economical to use hydrogen as intermediary for electricity transport, as it can be moved through pipelines and ships and takes its loss mostly in the conversion, not the distance moved. Other benefits are the ability to time shift loads and to use the hydrogen directly in industrial processes.
> The Changji-Guquan ultra-high-voltage direct current (UHVDC) transmission line in China is the world’s first transmission line operating at 1,100kV voltage. The transmission line traverses for a total distance of 3,324km and is capable of transmitting up to 12GW of electricity.
Losses can be compensated for with...more renewable generation.
(note that the above distance is roughly the distance across Europe east-west, distance across the US is a bit more, but you don't need to pull east coast generation all the way to west coast load centers or vice versa)
Also, if you pay the cost of making the hydrogen, you can use it not only for transmission, but also for storage at both ends of the pipeline. This allows the pipeline to be operated at constant high load, allows smoothing of the production fluctuations, and smooths demand fluctuations too. Multiple bites from one apple.
The cost of a hydrogen pipeline is only slightly larger than that of a methane pipeline of the same BTU capacity, even though the energy value of a hydrogen molecule is considerably less than a methane molecule. That's because hydrogen has considerably lower viscosity, which reduces pumping costs.
This is interesting. I guess I would've have expected more and more localized generation and coordination rather than long distance transmission and storage, but maybe it will be more of a combination of things.
Thanks for the insights, its not something I really thought of before.
There are already DC ties to the other grids in Texas. They do buy and sell power from other regions. They just aren't particularly large ties, maybe capable of 2GW total?
It's possible to still net lose money by exporting power. So much so that it might even still make sense to not do it even after a catastrophic failure every decade or two. The actual calculation will need to be done by those who have the figures and exact costs handy.
If not, what 'numbers or evidence' are you referring to?
The only thing I mentioned, costs and figures, are explicitly not known by me, so it seems bizarre to think I made judgements based on non existent knowledge.
Texas has an isolated grid. They can't buy or export power. This causes grid overloads. People have ended up freezing and dying in heat waves due to power outages.
You said "maybe it ok, maybe it saves money even if there are catastrophes, but I don't even know what the money is like".
Why would you say that a terrible power grid induces crisis is ok at all, let alone when you don't even know how the money breaks down?
Thank you. This is precisely what I wanted to reply but didn’t trust myself last night to not get in trouble with dang.
He is the one who specifically brought up profits as a potential reason to accept catastrophe then got all offended and aggressive, name calling etc when people challenged this. Even going as far as to deny bringing profits into it THEN admitting he had no idea of the numbers but STILL doubling down on the ‘don’t believe your lying eyes about what I wrote’.
I genuinely questioned whether it was just me until I checked back in the thread this morning.
Or at least type out the quoted sentences that you remember with 'edgy' and 'pathetic'? And then any passing reader can email dang to confirm if that's in a past comment revision, like you said?
If your unwilling to put in that basic effort after making such an accusation, then it seems self-defeating, plus no one would trust it.
I do sometimes use 'edgy', so maybe that is possible, but I really don't think I've ever called anyone 'pathetic', so this seems like a lame accusation.
(As a sidenote I don't even remember for my own comments made 48 hours ago how I edited them or what words changed after, so it's pretty astonishing that another user is keeping track.)
Well this seems a bit too snarky and not in good faith so I'm not going to substantially respond any further.
I wasn't too impressed with the last conversation I had with you, Cyberdildonics, and at best it seems as if your objecting to your own hasty paraphrase of my comment so I don't see how this could benefit any passing reader either.
All I see here is you avoiding backing up anything you are saying, even after you edited your comments.
You basically said 'what if it saves money' when discussing multiple state wide crises. Saying 'what if' isn't an argument in the first place, now you're avoiding confronting what you said completely.
These were your own comments, there is nothing rambling here. It is A to B to C. If you could confront the topic instead of avoiding it with names and insults I think you would have already.
You are also free to contact any HN user you trust and confirm whether I'm making this up or not?
I don't understand your question at all. Making what up? Why are all your comments deflection? Why not quote yourself and explain what you were trying to say?
Again, if you are unsure about or believe another user is somehow being duplicitous about HN norms, rules, mod decisions, etc., then double check, such as by emailing dang.
Frankly, although I didn't want to bring it up previously lest it be too embarrassing, perhaps logging off and seeking some help would be the better choice, if you are genuinely forgetting how to check past HN comments.
EDIT: After thinking it over, I'm going to disengage here since I rather not risk the chance of inadvertently nudging someone further into a negative spiral.
I'm not going to substantially respond any further.
You didn't substantially respond at all.
No one is victimizing you by asking you to focus on what you originally. You originally said that maybe statewide energy crisis that directly result in deaths are fine because it might make sense financially without even having any numbers or actual information of any kind.
All you did after was try to be insulting and act like you are being victimized while avoiding anything about what you originally said.
Why not just deal with the current topic? Why not explain what you said? It's bizarre that you would rather thrash around and try to blame people for replying when you could just deal with the thread directly.
Long-term trends clearly demonstrate the energy grid’s transition to renewable energy sources.
However, renewables like solar and wind come with unique challenges due to their intermittent nature. They are more variable, harder to forecast, have location constraints, and can benefit from battery storage. These factors lead to a more dynamic grid than before.
For instance, several regions in the country provide five-minute updates on their energy generation mix, enabling near real-time observations of renewable energy effects throughout the day
For example, numerous regions across the country provide updates on their energy generation mix at five-minute intervals, allowing for close to real time observations of these effect of renewables throughout the day.
To help those involved in the energy transition, I created an open-source project called Grid Status (https://github.com/kmax12/gridstatus) that provides fuel mix, wholesale pricing, load, load forecasts, and more.
Additionally, I've developed real-time visualizations to make this data more accessible and easier to comprehend: https://www.gridstatus.io
I hope making this data more accessible and understandable will accelerate the transition away from fossil fuels.
It demonstrates, quite clearly, that energy cost is not a concern. We can just barrel through to the electric revolution without a single care to cost, availability, or reliability.
One consequence of having many market participants is the availability of data that is published to make the markets function.
If you’re interested in seeing more about the real-time operations, I built a site that tracks all this data: https://www.gridstatus.io/live