The Reality of APIs in AgTech

Aug 16th, 2023 - My 39th Edition of Easy Observations in Agriculture

Visit my website www.easyobservations.com for more info about me or follow me on LinkedIn and Twitter

Opening Thoughts:

Oh I’ve been wanting to talk about this one for awhile. I even put a poll out on Linkedin about it asking you what you wanted to hear about and this is it. Now before I go into detail I’m going to say this one thing, I’m 100% a proponent of API connections in AgTech and beyond. I want them to work as best as possible and also let’s be honest, the industry would sort of suck right now without them.

Of course in saying that Ag would suck without them it doesn’t mean they are perfect either. It’s really a love hate relationship in many ways and it’s one I want to dive into naturally. Also, if you’re wonder why I might know or be qualified to even talk about this I’ll just say it stems from my past workings and I’ve talked about those enough so just take it for what it’s worth.

What I’ve found over the years though is a growing kaleidoscope of methods and tactics in how groups in Ag have used it, needed it, or down right hated it. I really can’t emphasize enough that this growing craze of everyone connecting to each other in Ag via APIs is a giant farce. It’s wanted, but it also really can’t happen. At least not in the way we think we want it or how it actually works.

Alright, stop dilly dallying Nathan and get into already…

The Dream:

We are all part of it these days in the tech world. It seems everything is connected in some ways and it’s just a core function of technology from IOT and Software to how our Social interactions are bound to one another. APIs are a BIG deal and naturally so they should be in Agriculture. They are, but it’s taken a lot of haggling to even get where we are today.

Here is the old story of why people want it from farmers to corporates: The typical user of software has like 5-10 different softwares that do different things and don’t connect. People hate manually entering the same crap over and over again from one system to the other with simple data and proprietary types too. Yes, that sucks and it should be better I get it.

So that has been a real goal for a long time, but until the good ole internet came around and rural places got even a semi decent internet connection it wasn’t really a thing. It probably started to really show it’s face of possibility around call it 2012-ish if you ask me. Maybe it was because we accepted our fate of the Mayan calendar not bringing us a Doomsday event, or more likely it was because some money started flowing in and the people from other mature tech areas started flowing into Ag slowly. We may never know (it’s the later).

So all that being true, many took it upon themselves to create their own API hooks and documentation. There was a clear focus on why beyond the normal “We just want everything to talk to each other” stuff and that was expanding ones market and reach. The big timers were doing it too so it was just the thing that was going to happen. All these softwares and tools started to say just connect to our APIs and you’ll get our data and we can share in the pie.

Plenty of hope was there to do this and I’ve written about it before in why it’s very important. The great thing was it was even starting to happen, but of course with any new thing coming into an old industry you started seeing problems arise. Nothing new, but being that it was Ag kind of made it different. Why that is I guess you could compare it to the industry I always compare it to in Health Care. There just are so many variables at play and different cultural and even legal differences that it’s just really hard to do.

Another way of thinking of it is this: Tech in general has been using APIs for awhile and it generally looks to work pretty decently on the surface. The normal stuff you see though is naturally connected through core similar mediums with data that is typically easier to transfer say with letters and numerical figures. While that is true in Ag as well in many cases, Ag also has other complex datasets that are strictly unique to itself or also has datasets that are very “heavy” to work with. Imagery is a great example of that actually or certain machine data that is proprietary.

While the dream is wanted and needed, we also need to wake our damn selves up and realize why it’s hard and also not working the way we all dreamed yet if that is even possible.

The Farce:

Now here is where it gets interesting. I was thinking earlier of how can I explain this in the best and dumbest way possible that makes sense and also probably confuses someone. So Analogies are coming my peeps.

Before I do that I want to explain first why the farce exists in why API connections haven’t had the so called Pay Day we all have hoped, so far at least that is. It comes down to two things. Legacy vs Web-Based. I’m guessing most of you get this, but I want to use this stupid analogy as it helps myself I suppose.

Legacy systems to me are a hardened concrete buildings. It’s built to work on its own and last without much if any changes at all. The surrounding concrete basically protects the innards which would be susceptible without it. At first it was just the way to do it (think stone way back too) and also once you build it it just stands and is hard to remove, change, and go away. Stupid maybe, but this analogy plays in my mind.

Now we have Web-Based. It’s like your normal stick building or say tin shed. It’s easier to build, takes less time and money (more recently a thing), you can open and make new doors easy, and stuff like that but it is easier to get into and steal crap as well basically. It’s easier to manipulate and connect to other structures more or less.

So now comes Ag. From just how software used to be built, how our infrastructure just is (land and buildings), to how the machinery OEMs want to protect their stuff. We created a bunch of hardened concrete structures. Now comes more newer age software and we created all these simpler malleable buildings. Connecting them is a freaking hard effort. Sure, the simple building can create this side structure (api) easily to get to the hardened one but now we have to cut a hole in the it and open it up to the vulnerable insides. Hard and dangerous.

That is the problem in a nutshell. Ag has and did build these hardened things as it had to and the reason is somewhat simple. Ag needs to build things that last as our cycles are long and susceptible to things we can’t control. It’s the nature of what it is. Of course now everything is telling Ag to connect to it. That’s also the whole farmer privacy thing in a nutshell as well.

To call it out in the simplest terms of why all these connections are a somewhat lie is that we secretly don’t want to connect to anything and we want to control what we have control of. That is how Ag has always been. Scared of either competition, losing control, or the unknown.

Now if I will rebuttal myself, we also do want to connect and open ourselves to certain things too. The thing is we only want to do that with things that have power and that actually will make whatever we have better. No one wants to connect the something that is not popular or that won’t actually bring them much value no matter how easy it is. There is a reason why everyone in AgTech wants to connect to let’s just say two groups. It’s because they have the users and power for now.

Turn that around and do you think the most powerful and popular groups what to connect to you? Probably not, unless whatever you have is of very specific and of powerful value as well. You see it’s just not about connecting things because some want it, it needs to make actual sense. Just because the 5 or 10% of your users want connections to 10 others doesn’t mean it makes sense. That group will always exist. The only way it makes sense is if 80% of them want it.

Without it, sorry, it’s a waste of time usually.

Money Honey:

Now that I’ve described my basic and weird thoughts on why the dream is great and the reality is sort of false I’m going to dive into the one that really tears it into a new a**hole. Cost….

There are a few groups (my former employer even) that will suggest using their APIs to connect to other APIs makes sense. In the simplest of terms, that is correct 100%. Except it also isn’t. Why it’s correct comes down to one maybe two main arguments.

The first argument is maintenance. It’s one thing to connect to one or two APIs, but also typically you will connect to the most complex ones first as they are also the most popular and valuable. Those groups being complex have a lot of maintenance within themselves so actually dealing with the typical changes can really mess things up. If someone else is taking care of that then there could be value there.

The second argument is that while most of these APIs are free to connect to, they are often not unified where they can be so while trying to bring one in is one experience, the other is totally different even if they are both sending over very similar data. It could be faster and easier to connect to a unified approach and typically is if you actually time out the man hours to do it and also add on both arguments together.

So, you should do it right? Maybe, but that’s not the core issue. Let me break it down financially I guess to really hone in.

In my past time at AgIntegrated it basically worked as so… We had a like 5ish things that brought together multiple APIs or data connections into 1 APIs to make it easier. From other APIs like MyJD and Climate to multiple soil labs to data translations stuff. Each one cost anywhere from 10-15k a year for unlimited use beyond some side use cases where we had to charge more. At the peak there was like 70 clients (give or take probably 10 I guess) using these APIs from startups to corps and softwares to ag retailers. That’s the base.

Out of those 70 there was only like 15 maybe 20 that used the full suite of APIs and naturally you add in some discounts so let’s call it around 50k per year for the suite. Like 5 that were big we could charge double that so 100k let’s say. All in all, potential and seemingly a decent business model. It was, it got acquired. Sounds great but also can you see the problem?

I won’t get into exact numbers but you can sort of add what you want and the reality is there is say 1.5-2m sitting here in revenue. Great and all, but we are talking about 70ish groups. That was basically all there was and could be for potential clients. At best you could say there was 100, it’s less now because of M&A and the buyers usually were already clients anyway. Call it a wash if you want in some ways. Think about it though, there is only at best 100 groups to connect to via APIs in AgTech that are in any realm big or important enough in some way to have some perceived value. That’s a horribly low number compared to other tech industry types. It’s actually probably lower, but you also could say higher if you expand into Agrifood I guess. Flip a coin and it’s still hard.

Can you charge different or use a different model? Maybe and we thought about it and tried going transactional in some ways. That makes it even weirder in some sense as it would limit the thought that the smaller groups could expand fast and large without crazy costs. Also, not all data via transactions is of the same value so it was easier to do a blanket cost. You win some you lose some basically. It’s still the wild west on that.

So what else then? Well, you can have all these connections or products to make it easier but many times the other software that wanted a connection only wanted the popular ones like MyJD or Climate. That’s all as I explained why above. All the other connections were and still are nice to haves or are only needed for niche areas really and that might be only 10-20 of those groups. What happens and happened many times is that the potential customer would just say screw it and do it themselves as they wanted….control. That old word I mentioned earlier.

While on the surface it was likely better for us to be the medium between one API and another because of upfront cost and maintenance, it didn’t solve the problem on control. Most AgTech softwares are very different to the next and control is huge because it’s all different. That is also another issue, we have basically no standards in Ag Data and while it’s been tried it hasn’t been enough. It’s honestly maybe the only way it can work.

As you maybe are starting to see, it’s not black and white to connect one to another. The groups that are trying to do these similar things have all of the problems above in common for the most part. How do they make it one might ask? You add other custom services to help the larger ones actually do the work that they would have to do themselves for cheaper. Custom Dev Services. That is the extra play and a viable one. Then again does that really make the process better when the pool of potential clients currently is super low? Sure…

Well what now?

By now I know there’s some out there rolling there eyes and getting ready to write me a concerning response. It’s always a thing, but usually that thing is deleted and it comes back to “Dammit, that is the way it is…shit.” I’ll gladly take criticism and likely deserve it. That is not my point here to make APIs and the great data connection points they have turn into a bad thing. It needs to happen and is viable to a point. So then what do we do to solve all the issues that I’m addressing?

Well…it’ll take two really really hard things and that is incentive and work. We need standards even though we have all this old crap legacy stuff. We want control, but we need trust. We collectively are letting down each other in not making it easier to work together. Money in this whole thing is really the problem, not enough to do the tech development needed and not enough of it to spend the extra time time to work together and figure it out business wise.

I know the industry wants and needs it. We all do. We are also relying on only a few to do all the work and take all the risk. To be honest, if you want to solve the problem you basically need the USDA and other governmental institutes to say heres a bunch of money and figure it out with “these” basic rules. It’d help everyone. Then again we all know how regulations and government can and does typically work perfectly.

Overall, I don’t know the answer beyond we will likely be stuck in the same pattern. The money in AgTech that needs to do this is declining right now so yeah get used to it. There are some leaders in the space, but damn it’s hard. I want to help and will where I can.

The biggest thing maybe now is to accept the reality and slowly but surely find a way to mitigate it in the future.

In Closing:

I’ll continue more or less of what I’m saying in the last section. Yes, it’s a shit show really to try and connect to others. Most groups that want to are also too small and don’t have the resources to really do it. It’s not just connecting this and that. It actually typically can take 6 months to a year implement an API from another and then that other group also has to worry about that connection too which is time and money. It’s tough.

I’ve literally dealt with this from both sides and I can tell you it sucks. Why does it have to suck and why is it still so expensive? I’m not exactly sure beyond it’s new and we have to fight through it all. Yeah stupid but real.

From here I’ll close with saying one thing, we need to get out of our legacy thoughts, wants, and needs and start thinking about unifying our larger goals because trust me when I say there are outsiders that likely will force it without letting us to do it the way it wants, needs, or should be done.

It’s hard stuff to do and while it’s far from perfect now we can’t let it get worse than it is. We need to find a way in some way to make the way to connect better for all. Peace

Thanks for reading Easy Observations! Please Share, Subscribe, and Comment if you would be so kind. I’d also be happy to get together and meet if you’d like to talk more in-depth about AgTech or Earth Observation in Ag. Feel free to contact me at my website. All the best!