The dea(r)th of Oracle RDBMS and contracting?

I’m feeling flat today and apologise for the sensationalist headline.

I also apologise for another one of this useless opinion-based posts. There are far too many opinions in this site, not to mention far too many posts on things requiring further investigation which never gets done, and not nearly enough concise, factual posts which give anyone any genuine interest. One day this will change… but not today – it’s cathartic to let it out.

So, I tend to contract these days, specialising in pretty much anything Oracle RDBMS-wise coming primarily from a development/application angle – i.e. development, design, performance tuning, architecture, development DBA.

Two observations I have about the current market for Oracle contracts in the UK.

Firstly, an increasing number of clients have HR-imposed limits where they will engage a contractor for a maximum of one year. For a database specialist, this presents a bit of a problem. So much of what we do is about the data. I find it takes at least six months to get a really good handle on just a fraction of the business and its data. So, after one year you’ve not long started ramping up the value that you can deliver. Maybe this behaviour is more common in banks and other such finance companies but, apart from my current client, I’ve talked to two prospective future clients who have the same such policy. This sort of policy only make sense if you think you can swap in/out any development resource and put no value on them having knowledge specific to your business.

And now to Oracle. I feel like the war has been lost and there are only a few pockets of resistance left now, resistance that will sooner or later be squashed. The religious war regarding sensible, about pragmatic use of databases and database code, about doing work related to data in the database, about data quality being enforced in the database, etc versus the database should be a bit bucket camp. 

Over the last few years I’ve worked on some pretty decent databases, some of which I created. They all took a pragmatic approach to the database. Even in n-tier Java environments, if it made sense we put the logic where it made sense, where there was a strategic or performance advantage whichever tier that was.

Even at my current client, I was pleasantly surprised to find a database-centric application – you might even say excessively so. But not for much longer. The database is under attack. A newly created hierarchy have decreed that databases are indeed bad.

And I was speaking to a friend today at a previous employer, a major media / entertainment company. They are planning to abandon their pragmatic approach to Oracle and switch wholely to open source databases, ORM tools, and the like.

I just don’t understand why. Actually I do understand some of the why. 

  • Databases don’t perform well when SQL is written by people who neither like nor understand SQL, people who don’t appreciate that they need to abandon their iterative approaches and need to think in sets, people who struggle to string together a couple of tables.
  • SQL doesn’t always perform well when written by people who neither like nor understand access paths and indexes.
  • It’s difficult to write good SQL against poor table design.
  • When databases don’t perform well, people don’t want to wait for people to tune or redesign, they want to buy some more memory or CPU and have it installed later that day. 
  • People of Influence with a decent database background are becoming few and far between.
  • People are reluctant to use built-in features like RLS, Oracle Audit Vault / FGA, etc, etc and prefer to write their own framework from scratch.
  • And not insignificantly, database testing tools are way behind the curve. Managers are getting used to full-featured testing reports, code coverage and code metrics and rightfully see the database as backward in this area.

And if you’re going to have a bit bucket, well, you might as well have a free bit bucket. And then you might as well stick ORM on top of it.

34 Responses to The dea(r)th of Oracle RDBMS and contracting?

  1. forkboy says:

    I think you’ve hit something there. The reason is likely because the universities are pumping out java drones. They know java, so the more they can do in it the happier they are. With application servers having built in horizontal scalability,if your application doesn’t perform, you just keep adding hardware and caching layers. Never mind that you likely could have done it all with a 10th of the hardware if things were done right.

  2. rc says:

    One of the problems is that we use always old versions of the database. I develop for Oracle 9, I can only develop for Oracle 10 when my last customer migrates to Oracle 10. I still have to wait one year.

    For .NET (middel tier and client) it is different. We use .NET 3.5, Microsoft has released 3.5 in november 2007.

  3. yadba says:

    amen 😦
    i did not get it either… it is all so wrong, wrong, wrong!

  4. intersting post, i’m feeling as you. Software development seems moving toward database agnostic-software and i agree with almost your points.

    Cheers,
    Cristian

  5. chet says:

    It is amazing isn’t it?

    I don’t know the reason for it, I wish I did though. It would probably aid me in fighting it at my current employer.

    My theory is that app dev people don’t understand (as you mentioned) the database, thus can’t make it work right. Now these people are in positions of influence and they want to make it as easy as possible for them and their cohorts.

    Our lead architect told me 4 times that he had 29 years of experience…so that obviously means he is right and I am wrong. It’s probably a middle ground (that tilts heavily to the database) somewhere there…

  6. Tal Olier says:

    I very much liked your writing, but totally disagree.
    Practicing for the last 10 years in Oracle and SQL Server in very large organizations I learned that large organizations will never use open source RDBMS for mission critical applications because (1) Security issues – the code is open (2) No real contact support when a problem found.

    –Tal.

  7. dombrooks says:

    I hope you’re right Tal, I hope you’re right…

    I think what you’re saying definitely was true but it seems to me that the tide has turned and it no longer applies to the same extent.

    I also suspect that the number of problems that are addressed by official support is dwarfed by the number of problems caused by people who don’t know what they’re doing.

  8. Pingback: Doug's Oracle Blog

  9. chet says:

    Tal,

    I would disagree. We’re (or were recently) a Fortune 500, but we’re using MySQL. HIPAA? SOX?

    Still, the march is on and there seems no stop to it…

  10. Peter Scott says:

    Dom, in away I am lucky, the sort of databases I work with (large data warehouses) need to use enterprise class features to work well, and I don’t often come across developers trying to emulate a data warehouse in some form of vendor-neutral-lowest-denominator client app. But move into a transactional world and horrors you see.

  11. dombrooks says:

    Thanks Chet – I know from your posts that we often seem to fight many of the same battles.

    Like you and your Fortune 500 place, I’ve worked for several FTSE 100 companies and this does not seem to be a barrier to abandoning commonsense.

  12. dombrooks says:

    Thanks for your comments Pete.

    I did imagine that was largely an OLTP phenomenon.

    Maybe it’s time to concentrate on data warehouses and BI as per Tim Gorman’s anecdote on Doug’s blog. Maybe that is a bastion of relative sanity? Data Warehousing and BI, that is – not Doug’s blog 😉

  13. Doug Burns says:

    Maybe that is a bastion of relative sanity? Data Warehousing and BI, that is – not Doug’s blog 😉

    I agree.

    a) DW is a relatively sane habitat for a database person.
    b) My blog isn’t.

    P.S. You’ll need a broomstick and a cauldron first, though, but Pete should be able to sell you one of his old ones.

  14. chet says:

    The weird thing is, this is definitely driven from our leadership. The architects are merely doing what they are being told to do…

    I think it’s part of the legacy they are trying to leave. While it might look good on the bottom line in the short term, I fear for the long term.

    Actually, the long term will probably be good for me, as I’ll always have plenty of work to do rewriting applications!

  15. dombrooks says:

    Oh yes Chet, that reminds me – that’s because leadership go on far too many “new paradigm” seminars and get impressed by people whose qualification is that they have written a book.

    I think I might write a book called “anyone can write a book”…

    Seriously, as a community we don’t do enough to highlight the cost savings, the time savings and the efficiency of doing things in the database. But even if we did, they’re not trendy enough to be attended.

    Maybe Oracle as a company/vendor should have done more? But then with their whole Fusion offering they have feet in both camps so probably don’t mind the net affect?

  16. Doug Burns says:

    Well you know where I work at the moment Dom, and they have new methodologies coming out of their ears. Fortunately, they might like to shout about them but, when you look at the systems the company truly depends on, it’s large swathes of PL/SQL, written by people who know what they’re doing, that makes them tick.

    Oh, and all developed before Agile turned up.

  17. Chris says:

    Personally I’ve come accross the HR issue of limits on contracts only recently having just moved from being a permanant employee. The issue goes further than just a limit on contract length there is a genuine belief that you can swap any developer/DBA/technician with a replacement and loose nothing of value to the company. Usually this is driven by percieved cost savings, by not rewarding technical excellence more experianced (and expensive) people leve and are replaced by cheaper staff even if in the long run it means higher costs due to poor software or missed deadlines, these things are harder to measure than just adding up all the salaries..

  18. chet says:

    I’ve been wanting to do something showing the cost savings of putting it in the database, at the very least, I could have facts as opposed to just a feeling. That might go a lot further with the leadership than just saying it.

    Do you know of any such document that exists?

  19. dombrooks says:

    I’m not aware of one, Chet.

    But whilst on Wednesday when I wrote this article I was feeling beaten, today I’m feeling a little more belligerent – helped not least, by the response to this and the knowledge that there is a safety net in the more data-centric haven of Data Warehousing and BI.

    But maybe what we need is a set of benchmarks(or challenges) for a set series of tasks to see how different approaches compare for initial outlay, lines of code (proxy of time to develop and complexity?), speed, resource consumption, etc, adapted from the TPC benchmarks perhaps…

  20. Fred Allison says:

    In the end it will probably be Oracle’s licensing policies that will force using alternative solutions. We have spent 4 years tuning and optimizing both our application and our database to support a very high volume web site. We are in the process of setting up a remote site for disaster recovery and have discovered that we will be on the hook for over 200k in Oracle licenses for a site that will never be active unless our primary data center is taken off line.

    We’re looking at replacing our 5 year old Sun server with a Dell 4 socket that uses the Intel E7330 processor. It appears that list price for Oracle Enterprise edition for that box, which costs less than 23k, will be 320k. Kind of hard to talk a bean counter into that, no matter how well it performs.

  21. dombrooks says:

    Thanks Fred.

    I certainly see that as a logical end point to moving towards a bit bucket database.

    Once you’ve moved all code out of the database, why pay for it? I for one couldn’t justify much of that cost on just the value of Oracle support.

    The cost of licensing is certainly an issue and one which has been done by others elsewhere – particularly related to some of the ADDM/AWR functionality.

    Licensing cost is one of the reasons why I got worked up about the plans of my current client to go to RAC when it is, IMO, unsuited to the type of application that would be running on it. Fortunately, they have just abandoned the plans to go with an initial one node RAC cluster. It’s needless expense and while the people signing the cheques are not currently bothered about it, how long before someone turns around and asks how come Oracle is so darn expensive? (And the db would probably be a bit bucket by then in our application)

  22. chet says:

    I wouldn’t begin to know how or where to measure this. I have talked to a friend at work who is a recent MBA recipient. He was excited about doing that kind of research. Maybe I’ll just follow up with him again and see what is possible.

    Here are some random thoughts on how to approach this:
    1. How do you convince leadership that the bright and shiny is not always the best.
    2. I see it as a “return” to fundamentals. Simpler is better but everything our architects create is the opposite. Back to #1, how to convince leadership?
    3. As Mr. Kyte asks, what happens when the application language changes? You end up rewriting the entire thing right? Our architect said that UML tools would then generate the code based on the model in whatever language you want…

    Well maybe not approaches, just more ranting. Your belligerence is shared.

    I do try and take on all the tuning opportunities I can. I just need a clone or a like minded individual to help. 🙂

  23. RC says:

    I hope that Oracle will add more different types of constraints on the database.

    We now have primary keys, unique key, foreign keys, check constraints, not null constraints and unique function based index constraints.

    I hope more will come.

  24. Pingback: Oracle Musings » The Value of Information

  25. robert engels says:

    You people miss the point completely. As a person who writes n-tier code that is database agnostic (but comes from a group-up database background), the biggest problem is that every database stored procedure dialect if different, making supporting different databases VERY difficult (aka expensive).

    We use a hybrid solution – an ORM, but standard SQL (using JDBC) when necessary.

    Also, there are MANY instances where an n-tier solution will perform BETTER than a database centric one. With intelligent caching the tiers remove the bottleneck that the database becomes (with heavy stored procedure use).

  26. robert engels says:

    To clarify, without being able to support different databases easily, you end up with vendor lock-in, driving up the prices.

    Our company wrote our original applications using Oracle centric code (lots of stored procedures).

    The new applications are 100% database agnostic and it has allowed us to very cheaply get customers with otherwise could not have.

  27. dombrooks says:

    Thanks for your comments, Robert.

    Not sure who you mean by “you people”. I for one have years of development experience in all tiers of n-tier applications, including solutions using ORM. But for all that, my current focus and main interest is databases.

    I can recognise that ORM does a job a large part of the time for those who want to use it. I’ve also witnessed the big downsides of ORM generated code when it goes wrong.

    For database-agnostic solutions, I don’t have a problem with ORM. There’s no point in having one version for Oracle, one version for SQL Server, etc, etc. I wouldn’t disagree with you at all in that regard.

    For single-vendor-database solutions, I see ORM as the first port of call for people who can’t be bothered with the database.

    My main point regarding ORM for database-agnostics is that a large part of data-centric applications do not warrant a data-agnostic approach – a vendor application which has to support a variety of client databases – sure; a data-centric application where there’s more chance of the n-tiers changing than the database vendor – nope, sorry, I can’t agree.

    You say that there are many instances where an n-tier solution will perform better than a database centric one.

    That is true BUT I don’t agree that “n-tier” and “database centric” are mutually exclusive – you seem to. Most of the successful applications that I have worked on would be described as n-tier with data-logic and data-intensive operations in the database where it makes sense.
    I’ve worked on plenty where performance issues were a nightmare to track down, I’ve worked on plenty where the database had issues (sometimes caused by the db, sometimes caused by what applications or connection storms were doing to the db), I’ve yet to work on one where the properly designed database was itself the bottleneck, heavy stored procedure use very much included.

    And for absolutely top end solutions, I can’t disagree that caching tiers become significant.

    However, most of the time people reach for the cheque book to scale out the middle tier because they’re not interested in doing the basics well in the database.

  28. dombrooks says:

    Regarding your second comment, which I got after my response to your first, I reiterate that I agree with you 100%.

  29. Pingback: Log Buffer #90: a Carnival of the Vanities for DBAs

  30. chet says:

    @robert

    Am I one of “you people?”

    I was not talking about database agnostic solutions. Of course you don’t want to rewrite in PL/SQL, T-SQL or vice versa. I thought that went without saying.

    My main point though is that the majority of appdev people can’t be bothered with the database. If Oracle is your solution, use it. If SQL Server is your solution use it. It’s well beyond a bucket.

  31. Sheeri says:

    dom said:
    For single-vendor-database solutions, I see ORM as the first port of call for people who can’t be bothered with the database.

    And I completely agree. What I take issue with is the fact that the original post conflates “going open source” with “using an ORM” and “poor table design.” Last I checked, open source databases did not have the market on developers doing poorly when doing data architecture and SQL query writing.

    Yes, there’s a lower barrier to entry with open source databases, but I’ve seen poorly written queries and dismally designed schemas in Oracle and SQL Server as well.

  32. dombrooks says:

    Thanks for your comments, Sheeri.

    I agree -fixing poorly written queries and dismally designed schemas in Oracle is how I earn my living. From what I’ve seen, I’m not sure the barrier is any higher for Oracle than anything else.

    My article centres around using the Oracle database as a bit bucket but my arguments for data-logic and data intensive operations apply to any database, open source or otherwise. That it what I am most defensive about here – proper use of the database. I do have a vested interest in Oracle in particular – it being one of my principal skills and the theme of my blog articles.

    I would link “going open source”, with “using an ORM” and “poor table design” but not as you suggest I’m saying.

    I only mention “going open source” as a natural conclusion to dumbing down your database.

    Not because open source database are necessarily any less capable, but because what starts with developers poorly educated in the ways of table design, neglecting database features and using ORM, results in using the DB as a bit bucket and when you’re there, then there’s surely no point in forking out for Oracle or SQL Server.

  33. Pingback: The DBA is dead | EvDBT

Leave a reply to chet Cancel reply