[00:00.000 --> 00:09.000] Thanks very much. I leave you in the capable hands of Karen Jax. [00:09.000 --> 00:22.000] Hi, thanks, Jimmy. Hi, so I'm Karen Jax and I'm going to talk to you today about DBA evolution [00:22.000 --> 00:27.000] or the changing role of the database administrator. So you can see that my title at the moment [00:27.000 --> 00:32.000] isn't database administrator, but I was a DBA for over 20 years. [00:32.000 --> 00:36.000] I'm not going to talk through the agenda. I've just left it there so that you can keep track of where we're up to [00:36.000 --> 00:40.000] and when I might stop talking. [00:40.000 --> 00:45.000] So I promise that this would be a talk for everyone that's pondered life's important questions. [00:45.000 --> 00:50.000] What does a DBA actually do? Why are DBAs always so grumpy? [00:50.000 --> 00:54.000] How's the role of the DBA changed over the last couple of decades? [00:54.000 --> 01:01.000] What's the DBA of the future going to do? And will autonomous databases finally have put us all out of work? [01:01.000 --> 01:07.000] So this year marks the 25th anniversary of the start of my career as a DBA, [01:07.000 --> 01:13.000] so I thought that I'd allow myself a bit of a self-indulgent retrospective. [01:13.000 --> 01:18.000] So first, the evolution of this particular DBA, and I'm going to apologize to Claire at this point [01:18.000 --> 01:22.000] because this is going to be a bit of a me, me, me section of my talk. [01:22.000 --> 01:26.000] So I'm just going to have to get through that bit. [01:26.000 --> 01:29.000] Being a DBA definitely wasn't an ambition of mine at school. [01:29.000 --> 01:35.000] I mean, anybody here say database administrator when they were asked what they wanted to do when they grew up? [01:35.000 --> 01:42.000] I mean, I didn't even know I was going to go into IT until about a year before I started my career, [01:42.000 --> 01:46.000] although with hindsight it's fairly obvious that that's what I was destined to do. [01:46.000 --> 01:51.000] My dad taught me to count in binary before I could even properly count in decimal. [01:51.000 --> 01:57.000] He built me my first computer, an acorn atom, which he later replaced with a BBC Model B, [01:57.000 --> 02:02.000] and obviously I used the computer for playing Chuckie Egg and other games, [02:02.000 --> 02:06.000] but I also learned to code in BBC Basic. [02:06.000 --> 02:12.000] And I actually looked up the spec of the BBC Model B in case anyone's interested, [02:12.000 --> 02:19.000] so it had a 2MHz processor and 64K of RAM. [02:19.000 --> 02:25.000] I then went on to do a maths degree because I still had absolutely no idea what I wanted to do, [02:25.000 --> 02:27.000] and maths was my favourite subject. [02:27.000 --> 02:30.000] And after that, I figured I probably ought to get some real-world skills, [02:30.000 --> 02:33.000] so I did a masters in software development. [02:33.000 --> 02:38.000] Towards the end of that, one of the graduate schemes I applied to did some aptitude tests, [02:38.000 --> 02:43.000] and they came back to say that they thought I'd probably be good as a DBA. [02:43.000 --> 02:49.000] Once I'd asked Jeeves what one of those was, I thought that sounded like a fairly good fit, [02:49.000 --> 02:52.000] and that I'd give it a go. [02:52.000 --> 02:57.000] Fortunately, it turns out that Jeeves was right, and the aptitude test, [02:57.000 --> 03:01.000] and this is what my careers looked like so far, [03:01.000 --> 03:06.000] so I really haven't strayed very far from my DBA role. [03:06.000 --> 03:11.000] Obviously, the things that I do in my job have changed over time [03:11.000 --> 03:14.000] and I've gradually moved into more senior roles, [03:14.000 --> 03:17.000] and I've gradually changed the focus of what I do, [03:17.000 --> 03:20.000] but I've tried not to let that influence what I talk about too much. [03:20.000 --> 03:25.000] I'm trying to talk more about the DBA role in general. [03:25.000 --> 03:31.000] So, let's set the scene for the start of my career as a DBA in 1998. [03:31.000 --> 03:34.000] The 90s brought us Grunge and Britpop. [03:34.000 --> 03:39.000] They also brought us the first truly portable and affordable mobile phones, [03:39.000 --> 03:42.000] for example, the Nokia 3210. [03:42.000 --> 03:45.000] They brought us the first text message, the Palm Pilot, [03:45.000 --> 03:51.000] and also for better or worse, Amazon and Google. [03:51.000 --> 03:55.000] But what does a DBA actually do? [03:55.000 --> 03:57.000] Just so that we're all on the same page, [03:57.000 --> 04:02.000] I've taken highlights of definitions of a DBA from various places. [04:02.000 --> 04:07.000] I looked at careers services, DBMs vendors, and job sites. [04:07.000 --> 04:13.000] And the general consensus is that a DBA manages and secures computer systems [04:13.000 --> 04:17.000] that store data using specialist software. [04:17.000 --> 04:19.000] So, they're all pretty much agreed, [04:19.000 --> 04:26.000] but that doesn't really tell us anything about what a DBA actually does. [04:26.000 --> 04:32.000] So, first of all, one thing to note is that there are lots of different types of DBA. [04:32.000 --> 04:36.000] And the different types of DBA will be different from organisation to organisation. [04:36.000 --> 04:46.000] So, at ACME, for example, we might have the DBA role split into many different DBA roles. [04:46.000 --> 04:53.000] Whereas at Wonka Industries, we might have DBAs who are in one pool [04:53.000 --> 04:58.000] who are all responsible for all DBA tasks. [04:58.000 --> 05:04.000] A traditional split is between the production DBA and the development DBA. [05:04.000 --> 05:08.000] So, the production DBA will be responsible for keeping all of the databases [05:08.000 --> 05:10.000] in the production environment up and running. [05:10.000 --> 05:14.000] So, looking at things like availability, performance, security. [05:14.000 --> 05:18.000] The development DBA would work more closely with the developers. [05:18.000 --> 05:22.000] So, they've got a focus on building and maintaining the database environment [05:22.000 --> 05:29.000] that's supporting the application development lifecycle. [05:29.000 --> 05:31.000] Just wait for the photos. [05:31.000 --> 05:37.000] In some organisations, you might see a split of system versus application DBA. [05:37.000 --> 05:45.000] So, here the application DBA will be more looking at the logical application related aspects of the database. [05:45.000 --> 05:50.000] Whereas the system DBA is going to be responsible for the underlying software [05:50.000 --> 05:54.000] and the physical infrastructure. [05:54.000 --> 05:57.000] And the splits not clear cut. [05:57.000 --> 06:00.000] The roles often overlap. They may or they may not. [06:00.000 --> 06:04.000] And you might see lots of different types of DBA as well. [06:04.000 --> 06:11.000] So, you might have a data warehouse DBA who will look after the data warehouses [06:11.000 --> 06:16.000] particularly ETL from various data sources into the organisation's data warehouse. [06:16.000 --> 06:20.000] A cloud DBA I think I've got in there as well [06:20.000 --> 06:27.000] who will be looking after cloud hosted databases including liaising with the cloud provider. [06:27.000 --> 06:30.000] Also, you might see database architects. [06:30.000 --> 06:36.000] I know it doesn't have DBA in the title but it's a very closely related role and often overlaps. [06:36.000 --> 06:40.000] It's a role that's existed for at least a couple of decades [06:40.000 --> 06:49.000] but it's gradually become more popular and seems in a lot of cases to have split out from the traditional DBA role. [06:49.000 --> 06:54.000] And it's not an exhaustive list. You could see all sorts of different types of DBA. [06:54.000 --> 07:01.000] You might also get DBAs that look after just one particular task such as backup and recovery or replication. [07:01.000 --> 07:07.000] And any or all of those might be split into junior DBA and senior DBA [07:07.000 --> 07:11.000] with the junior DBA doing a lot of the day-to-day tasks [07:11.000 --> 07:15.000] and the senior DBA having more of a strategic focus. [07:15.000 --> 07:25.000] Okay, so we saw the definition of a DBA role [07:25.000 --> 07:29.000] but as I said it didn't really tell us much about what a DBA actually does. [07:29.000 --> 07:32.000] So what does a DBA do in 2023? [07:34.000 --> 07:38.000] I looked at the different definitions that I showed you earlier [07:38.000 --> 07:41.000] and there were lists of responsibilities that went with those [07:41.000 --> 07:44.000] and I've taken information from different job adverts [07:44.000 --> 07:48.000] and tried to pull together the list. [07:48.000 --> 07:53.000] So most people seem to agree that a DBA will do some or all of [07:53.000 --> 07:59.000] designing, implementing and managing backup and recovery policies [08:01.000 --> 08:06.000] designing and implementing security policies and managing database access [08:06.000 --> 08:11.000] although in a lot of places the security role has been split out into a separate security team these days. [08:11.000 --> 08:16.000] Implementing monitoring and ongoing monitoring of the databases [08:16.000 --> 08:22.000] design and development of the database including data modelling [08:22.000 --> 08:28.000] or maybe just reviewing data models that have been created by the development team. [08:28.000 --> 08:35.000] Support and troubleshooting including often being on a 24-7 on-call rotor [08:35.000 --> 08:42.000] and database software install and upgrades listed in a lot of them as a specific task [08:42.000 --> 08:49.000] but in a lot of places that's automated or the installation side of things is automated [08:49.000 --> 08:53.000] so the DBA would either be responsible for implementing that automation [08:53.000 --> 08:56.000] or another team would do the automation [08:56.000 --> 08:59.000] and the DBA wouldn't actually have anything to do there. [08:59.000 --> 09:02.000] Upgrades I would say it's more of a strategic role [09:02.000 --> 09:08.000] in working with the application team to make sure that upgrades are tested correctly [09:08.000 --> 09:10.000] so that they're successful. [09:10.000 --> 09:16.000] And there's a whole other page, a whole other list of responsibilities [09:16.000 --> 09:19.000] so providing database expertise to other teams [09:19.000 --> 09:23.000] performance tuning enhancement often troubleshooting [09:23.000 --> 09:29.000] capacity planning creating databases including for development and test environments [09:29.000 --> 09:34.000] database maintenance and data protection and GDPR considerations. [09:36.000 --> 09:39.000] So that was a massive, oh sorry, before I move on [09:39.000 --> 09:41.000] is a massive long list of responsibilities [09:41.000 --> 09:45.000] so clearly in most cases one person can't do all of those things. [09:48.000 --> 09:53.000] This chart shows some of the most requested skills for a DBA in the UK [09:53.000 --> 09:58.000] I've taken this from IT Jobs Watch so it's for the six months up to January 2023 [09:58.000 --> 10:05.000] so we can see that SQL is the first requested skill [10:05.000 --> 10:11.000] and I went up to number 23 because there we've got PostgreSQL listed there as well. [10:11.000 --> 10:16.000] On the next page I've summarised those different responsibilities [10:16.000 --> 10:19.000] sorry the different skills that are being asked for. [10:21.000 --> 10:25.000] So consolidating the list of skills it boils down to [10:25.000 --> 10:33.000] SQL so writing SQL, making it more efficient [10:33.000 --> 10:36.000] potentially knowing procedural languages [10:36.000 --> 10:39.000] integration and everyone's favourite ORMs. [10:41.000 --> 10:46.000] One or more database management systems usually still asking for relational. [10:49.000 --> 10:54.000] Knowledge of operating systems so this will be traditional operating systems [10:54.000 --> 10:57.000] but also various different cloud environments. [10:57.000 --> 11:02.000] Performance tuning, database migration seems to be quite a popular thing [11:02.000 --> 11:05.000] people are always moving from one database to another. [11:05.000 --> 11:10.000] Disaster recovery, social skills is listed most of the places as a separate thing [11:10.000 --> 11:12.000] I don't know what they're trying to say there. [11:16.000 --> 11:20.000] HA replication clustering type skills [11:20.000 --> 11:25.000] and then not listed in any of the things I found as specific skills [11:25.000 --> 11:30.000] but most DBAs I know are actually expected to know about different DevOps tools [11:30.000 --> 11:33.000] and methodologies and they're expected to know how to use automation tools [11:33.000 --> 11:35.000] like Ansible or Puppet to do things. [11:39.000 --> 11:43.000] So how does all that compare to what a DBA did in 1998? [11:44.000 --> 11:46.000] On paper it actually looks very similar [11:46.000 --> 11:50.000] so there are just reproduced the same list of responsibilities as I had [11:50.000 --> 11:52.000] for what a DBA does in 2023. [11:52.000 --> 11:55.000] Although I've taken off GDPR because I didn't exist yet. [11:57.000 --> 11:59.000] Does that mean nothing's changed? [12:00.000 --> 12:06.000] Well obviously although the responsibilities are pretty much the same [12:06.000 --> 12:12.000] the skills that a DBA needs and the tools that a DBA is expected to be able to use [12:12.000 --> 12:17.000] have changed massively because a lot of the tasks that they do day to day [12:17.000 --> 12:19.000] are done very differently. [12:20.000 --> 12:24.000] And obviously that's not limited to DBA tasks or even IT tasks [12:24.000 --> 12:27.000] we do loads of day to day tasks differently as well. [12:27.000 --> 12:30.000] So I've used send my sister a birthday card as an example [12:30.000 --> 12:35.000] so that's the flowchart for 1998 I would have gone to the shop [12:35.000 --> 12:38.000] I would have browsed the racks, chosen a card I liked [12:38.000 --> 12:41.000] I would have paid for that card and probably bought a stamp at the same time [12:41.000 --> 12:43.000] taking it home, written a message in it [12:43.000 --> 12:46.000] put my sister's address on it, stuck the stamp on [12:46.000 --> 12:53.000] and then walked it to my local post box to be collected by the post delivery. [12:53.000 --> 12:57.000] I can still use that method in 2023 if I want to [12:57.000 --> 12:59.000] but this is what I'm more likely to do [12:59.000 --> 13:03.000] I'm going to open the app of my favourite online greetings card retailer [13:03.000 --> 13:08.000] I'm going to upload a photo from my phone, type in a message on my sister's address [13:08.000 --> 13:11.000] pay using my disposable online credit card [13:11.000 --> 13:15.000] and just get them to send it directly to my sister as soon as it's printed. [13:20.000 --> 13:23.000] So if we go back to DBA tasks [13:23.000 --> 13:26.000] I've taken installed database software as an example [13:26.000 --> 13:29.000] I've not put this there to be read [13:29.000 --> 13:34.000] this is an extract from the Oracle 7.3 installation guide [13:34.000 --> 13:37.000] which was my first database installation that I did [13:37.000 --> 13:41.000] I can't imagine that many DBAs in 2023 [13:41.000 --> 13:45.000] need to first walk to the machine room and put the CD into the server [13:45.000 --> 13:47.000] before they start their installation [13:47.000 --> 13:53.000] it used to take me I think two days to prepare and perform a single database install [13:53.000 --> 13:56.000] and we didn't have a repeatable process [13:56.000 --> 14:00.000] there was a silent mode that was supposed to allow us to script it [14:00.000 --> 14:04.000] but because none of the servers were installed using any kind of automation [14:04.000 --> 14:09.000] they had their own particularities so it didn't work and we didn't bother with it [14:09.000 --> 14:15.000] in 2023 a DBA would it be expected to know how to use an automation tool [14:15.000 --> 14:19.000] such as Ansible to do a database software install [14:19.000 --> 14:22.000] and they'd probably not just install a standalone database [14:22.000 --> 14:26.000] they can install the whole database infrastructure [14:26.000 --> 14:31.000] including high availability monitoring back up and recovery [14:31.000 --> 14:34.000] all in one command [14:39.000 --> 14:43.000] so what kind of skills did a DBA need in 1998? [14:45.000 --> 14:48.000] I can't reproduce the list of skills quite as easily [14:48.000 --> 14:52.000] because that has changed but a lot of the general skills were the same [14:52.000 --> 14:57.000] so you'd be expected to know SQL and probably a procedural language [14:57.000 --> 15:04.000] you'd be expected to know one relational database management system [15:04.000 --> 15:10.000] you'd probably be expected to know how to do Unix basic admin and shell scripting [15:10.000 --> 15:14.000] database performance tuning and disaster recovery [15:14.000 --> 15:20.000] which was probably calling someone to bring the tapes back on site [15:20.000 --> 15:28.000] so that's what a DBA was doing in 1998 and is doing today [15:28.000 --> 15:32.000] what will a DBA do in another 25 years time? [15:34.000 --> 15:37.000] well I'm going to hedge that question for now [15:37.000 --> 15:42.000] so before we start looking at what a DBA might do in 25 years time [15:42.000 --> 15:44.000] everyone likes a bit of buzzword bingo [15:44.000 --> 15:48.000] so I thought we'd have a look at the buzzwords, trends, hypes [15:48.000 --> 15:52.000] from the last few decades just to give us an idea of what's changed [15:52.000 --> 15:56.000] around our vintage DBA and what's still changing [15:57.000 --> 16:00.000] so the 1990s [16:00.000 --> 16:05.000] the 90s brought us the worldwide web first with painfully slow dial-up access [16:05.000 --> 16:10.000] and finally broadband internet which meant that remote work was finally a possibility [16:10.000 --> 16:15.000] for some DBAs even if it was just to avoid having to trek into the office [16:15.000 --> 16:18.000] when doing 24-7 on-call [16:18.000 --> 16:21.000] the concept of big data was introduced [16:21.000 --> 16:25.000] data warehouses became popular and we needed massively parallel processing [16:25.000 --> 16:30.000] MPP to analyse all of this data so teradata became popular [16:30.000 --> 16:33.000] and a teaser didn't come along until 2000s [16:33.000 --> 16:37.000] scrum started to creep into our normal everyday work process [16:38.000 --> 16:42.000] object-oriented programming and from their object-oriented databases [16:42.000 --> 16:46.000] were the next big thing in programming methodology [16:46.000 --> 16:49.000] everybody was talking about AI and machine learning [16:49.000 --> 16:51.000] even though they weren't new concepts [16:51.000 --> 16:55.000] with deep blue-beating Grandmaster Kasparov at chess [16:55.000 --> 16:58.000] we were blessed with PostgreSQL and Linux [16:58.000 --> 17:02.000] SSDs started to replace spinning hard drives [17:02.000 --> 17:04.000] spinning disk hard drives [17:04.000 --> 17:08.000] but that's still a gradual ongoing move [17:08.000 --> 17:11.000] and a lot of us missed out on New Year's celebrations [17:11.000 --> 17:14.000] because we were on call to make sure that planes didn't fall out of the sky [17:14.000 --> 17:17.000] and fortunes weren't wiped out due to the Y2K bug [17:20.000 --> 17:22.000] the 2000s [17:22.000 --> 17:25.000] we were introduced to blockchain [17:25.000 --> 17:28.000] Cloud was and still is one of the biggest buzzwords [17:28.000 --> 17:31.000] with a move away from the traditional data centre [17:31.000 --> 17:34.000] everything started to be provided as a service [17:34.000 --> 17:37.000] we had software as a service and platform as a service [17:37.000 --> 17:40.000] we were all expected to be agile to embrace DevOps methodologies [17:40.000 --> 17:44.000] and to implement continuous integration and continuous deployment [17:45.000 --> 17:50.000] the Internet of Things and social media contributed to a massive growth in data volumes [17:51.000 --> 17:54.000] we got Hadoop, Exadata, NoSQL and JSON [17:54.000 --> 17:58.000] as new ways to store, process, analyse and describe data [17:59.000 --> 18:02.000] BASE was proposed as an alternative to ACID [18:02.000 --> 18:07.000] favouring availability over consistency for distributed systems [18:07.000 --> 18:11.000] access to data on the move started to be possible [18:11.000 --> 18:14.000] with Wi-Fi and containers were introduced [18:14.000 --> 18:17.000] although they weren't at this point quite ready for databases [18:18.000 --> 18:21.000] I'm going to take a deep breath and drink some water [18:21.000 --> 18:36.000] OK, 2010s infrastructure joined the as a service party [18:36.000 --> 18:39.000] NFTs, non-fungible tokens were introduced [18:39.000 --> 18:42.000] we started to store our data in data lakes [18:42.000 --> 18:44.000] which sometimes became data swamps [18:45.000 --> 18:49.000] columnar databases, even though they were actually a concept from the 1970s [18:49.000 --> 18:52.000] became popular and joined the other NoSQL databases [18:53.000 --> 18:57.000] we started to avoid putting all of our eggs in one basket with multi-cloud strategies [18:58.000 --> 19:00.000] Oracle announced their autonomous databases [19:00.000 --> 19:03.000] which meant there was no, not going to be any need for DBAs anymore [19:04.000 --> 19:09.000] CloudNative was the next big thing with microservices, Docker and Kubernetes [19:09.000 --> 19:13.000] and the EU caused DBAs all across Europe [19:13.000 --> 19:19.000] massive headaches by introducing the General Data Protection Regulations GDPR [19:22.000 --> 19:26.000] the 2020s haven't all been a bed of roses in tech so far [19:26.000 --> 19:29.000] with a global pandemic and massive layoffs [19:29.000 --> 19:34.000] on the other hand, remote work has started to become the norm for a lot of us [19:34.000 --> 19:39.000] access to data on the move is getting faster with the introduction of 5G [19:39.000 --> 19:43.000] Zoom became a household name [19:43.000 --> 19:45.000] everyone's now heard of Master Don [19:45.000 --> 19:48.000] and apparently we're all going to be living in the metaverse [19:48.000 --> 19:52.000] and using chat GPT to write all of our reports and presentations [19:53.000 --> 19:55.000] and that's just the first few years [19:55.000 --> 19:59.000] so who knows what's going to have happened by the end of the 2020s [19:59.000 --> 20:16.000] so, we can look at a few ways that databases have changed in response to all of those things that have been going on [20:19.000 --> 20:24.000] this is a massively simplified view of the history of database management systems [20:24.000 --> 20:26.000] please don't quote me on this [20:26.000 --> 20:32.000] so in around 1980 the relational database pretty much [20:32.000 --> 20:36.000] started to replace the old navigational databases [20:36.000 --> 20:39.000] and it's been going strong since then and still is [20:39.000 --> 20:46.000] in about 2000, various post-relational or no-SQL databases [20:48.000 --> 20:52.000] became popular and are still popular [20:52.000 --> 20:59.000] and of course everything goes in cycles, the no-SQL databases were around before the relational databases [20:59.000 --> 21:02.000] just that they weren't called no-SQL databases at that point [21:05.000 --> 21:08.000] it's easy to think that relational databases are going away [21:08.000 --> 21:12.000] when we hear about all the different no-SQL databases there are and how popular they are [21:12.000 --> 21:18.000] this is a chart from dbengens.com from January this year [21:18.000 --> 21:22.000] and it shows that actually relational databases are more than twice as popular [21:22.000 --> 21:28.000] as all of the different types of database management system put together [21:30.000 --> 21:35.000] and their figures also show that the popularity of relational database management systems [21:35.000 --> 21:38.000] has stayed fairly static over the last 10 years [21:41.000 --> 21:44.000] we've got more variety of databases [21:44.000 --> 21:48.000] so in the 90s there was a handful of database management systems [21:48.000 --> 21:54.000] and organisations would usually use one of those for all of their databases or maybe two [21:54.000 --> 21:58.000] there are now over 400 different types of database management system [21:58.000 --> 22:04.000] most organisations don't restrict themselves to just one or two of those [22:04.000 --> 22:09.000] so you'll get not just within one organisation multiple database types [22:09.000 --> 22:14.000] but also within one application [22:14.000 --> 22:17.000] so that means a DBA might be expected to know [22:17.000 --> 22:20.000] not just several relational database management systems [22:20.000 --> 22:24.000] but also various different no-SQL types they might be expected to know [22:24.000 --> 22:29.000] about spatial databases, time series, graph databases or document store [22:32.000 --> 22:35.000] databases have got bigger [22:35.000 --> 22:39.000] so the terms big data and very large database were introduced in the 90s [22:39.000 --> 22:43.000] but they don't actually tell us very much about the specific volume of data [22:43.000 --> 22:48.000] the definition in most places I found of very large database [22:48.000 --> 22:52.000] is a database that stores so much data [22:52.000 --> 22:59.000] that its maintenance and architecture need specialist tools and methodologies [22:59.000 --> 23:06.000] so it's basically just a database that's so big that we haven't yet figured out how to manage it [23:06.000 --> 23:12.000] so in the 90s most people I spoke to seemed to think that a multi gigabyte database was fairly big [23:12.000 --> 23:17.000] these days people have multi terabyte databases and they're seen as fairly normal [23:17.000 --> 23:21.000] once you start to get multi tens of terabytes you might start needing [23:21.000 --> 23:29.000] some specialist methodologies [23:29.000 --> 23:38.000] in 2018 IDC said that the global data sphere was predicted to grow to 175 zettabytes by 2025 [23:38.000 --> 23:45.000] I haven't managed to find any later figures so I don't know if that's on course to still be the case [23:45.000 --> 23:52.000] so the global data sphere is apparently all of the data that's created, captured and replicated [23:52.000 --> 23:55.000] in core edge and end point locations [23:55.000 --> 23:59.000] and one zettabyte is a trillion gigabytes [23:59.000 --> 24:05.000] so for anyone like me who has difficulty visualising 175 trillion gigabytes [24:05.000 --> 24:08.000] apparently if you burnt that onto CDs [24:08.000 --> 24:13.000] you'd have a stack of CDs that either took you to the moon 23 times [24:13.000 --> 24:17.000] or around the earth 222 times [24:17.000 --> 24:26.000] so it doesn't look likely that there's going to be any shortage of data to manage any time soon [24:26.000 --> 24:28.000] databases have got more complex [24:28.000 --> 24:34.000] I suspect we've all seen an application architecture diagram that looks approximately like that [24:34.000 --> 24:39.000] we've got multiple database types [24:39.000 --> 24:45.000] not just within one organisation but also often for a single application [24:45.000 --> 24:48.000] we've got multi cloud strategies [24:48.000 --> 24:50.000] we've got containerised databases [24:50.000 --> 24:52.000] we've got distributed systems [24:52.000 --> 25:00.000] and we've got many interrelated components [25:00.000 --> 25:07.000] so how has the DBA adapted in response to all of these changes? [25:07.000 --> 25:11.000] the DBA has had to develop a cloud mindset [25:11.000 --> 25:15.000] I was really reluctant to embrace cloud [25:15.000 --> 25:18.000] I can't remember when I first heard the word [25:18.000 --> 25:22.000] but I thought it was just a hype that was all going to blow over [25:22.000 --> 25:26.000] whether we like it or not cloud is here and it's growing [25:26.000 --> 25:32.000] so we have to work with it [25:32.000 --> 25:39.000] so with the advent of cloud computing many DBAs are no longer actually responsible for managing on-premise databases [25:39.000 --> 25:43.000] the hardware and software is often managed by somebody else [25:43.000 --> 25:52.000] so the cloud DBA will need to have knowledge of probably multiple different cloud platforms [25:52.000 --> 25:58.000] and even organisations that keep their databases on-premises [25:58.000 --> 26:07.000] are often moving to a cloud-like architecture, a cloud-like model with them self-service databases [26:07.000 --> 26:09.000] the DBAs had to learn to be flexible [26:09.000 --> 26:14.000] so as I've said they would be expected often to know multiple different types of database [26:14.000 --> 26:17.000] not just relational but also no SQL [26:17.000 --> 26:22.000] they need to be able to work in lots of different database environments [26:22.000 --> 26:30.000] not only the traditional operating systems but also cloud and things like Kubernetes [26:30.000 --> 26:36.000] they're expected to know how to use DevOps and automation tools and methodologies [26:36.000 --> 26:44.000] and they've got to keep up to date with all the changes in all of these different types of tool and methodology [26:44.000 --> 26:51.000] the other side of the coin is that actually a lot of roles have been separated out into different teams [26:51.000 --> 26:58.000] and you might now have different teams that are responsible for infrastructure, for data, security and the application [26:58.000 --> 27:01.000] we already looked at the different types of DBA [27:01.000 --> 27:07.000] but actually there are a whole host of different roles related to database and the data in it [27:07.000 --> 27:12.000] and this is just a few of the ones that I've seen by scanning job sites [27:12.000 --> 27:21.000] all of these changes mean that the DBA has to collaborate with other teams [27:21.000 --> 27:28.000] so not just with other database experts but also with teams that support the entire application stack [27:28.000 --> 27:32.000] I often say to people that I've been forced out of my little database bubble [27:35.000 --> 27:39.000] and the DBAs had to develop a strategic focus [27:39.000 --> 27:45.000] so the DBA is more likely to be looking at the overall architecture of the database [27:45.000 --> 27:54.000] analytics and data processing, improving automated processes, collaborating with other teams [27:54.000 --> 27:57.000] to make sure they've implemented business solutions [27:57.000 --> 28:01.000] rather than looking at the day-to-day technical details of managing the database [28:01.000 --> 28:09.000] so it's a reminder that change isn't all bad [28:09.000 --> 28:15.000] I asked colleagues and contacts to tell me some of the DBA tasks that they used to have to do [28:15.000 --> 28:19.000] that they are glad to be rid of and these are just a few of the answers [28:19.000 --> 28:27.000] vacuum cron jobs, data file reorganizations, commuting to work [28:27.000 --> 28:31.000] controversial ones, some people probably do still have to do that [28:31.000 --> 28:38.000] writing complicated backup scripts and installing post-GIS via scripts rather than create extension [28:38.000 --> 28:43.000] so you've probably all got your own tasks that you are glad to be rid of [28:47.000 --> 28:52.000] so the million dollar question, a variation of the question is the DBA role [28:52.000 --> 28:58.000] obviously is asked constantly all over the internet and of course database conferences [28:58.000 --> 29:03.000] and probably more so since Oracle announced its autonomous database in 2017 [29:05.000 --> 29:12.000] is the DBA the clock winder of the future or the lamp lighter, the switchboard operator, the video rental clerk [29:12.000 --> 29:15.000] or probably more likely the COBOL developer [29:15.000 --> 29:20.000] because even if things change so much that modern databases don't require a DBA [29:20.000 --> 29:27.000] how many decades is it going to take before all databases are migrated over to this new DBA-less technology [29:32.000 --> 29:43.000] fortunately data from the US Bureau of Labor Statistics says that there isn't a shortage of DBA roles [29:43.000 --> 29:56.000] so we can see this goes from 2003 so 2003 is the first year in which those stats listed database administrator as a separate role [29:56.000 --> 30:03.000] before that I think it was computer scientists or I can't remember the exact role [30:03.000 --> 30:06.000] but there wasn't a specific DBA role [30:06.000 --> 30:13.000] from 2020 it's listed as database administrators and architects [30:13.000 --> 30:17.000] so that supports what we saw before about the two roles splitting out from the same thing [30:18.000 --> 30:26.000] so this is the total number of DBA's thousands across the left there in the US [30:26.000 --> 30:29.000] so we can see that it's going up slightly [30:29.000 --> 30:34.000] it's not growing massively but it's definitely a slight upward trend [30:34.000 --> 30:42.000] and the Bureau actually projects a 9% growth from 2021 to 2031 [30:42.000 --> 30:45.000] which is higher than the average of all other occupations [30:46.000 --> 30:47.000] so that's good [30:49.000 --> 30:56.000] I tried to find similar stats for the UK but unfortunately the lists of job roles are more generic [30:56.000 --> 30:58.000] they don't have database administrator listed [30:58.000 --> 31:06.000] so instead I've gone to IT Jobs Watch and I've found the numbers of job adverts that have DBA [31:06.000 --> 31:08.000] that are for DBA or database administrator [31:08.000 --> 31:12.000] and those numbers are looking pretty healthy for 2022 and 2023 [31:15.000 --> 31:17.000] well how about automation? [31:17.000 --> 31:20.000] surely once everything's automated we don't need DBA's [31:21.000 --> 31:24.000] so one of my friends is a pilot and people like to ask him [31:24.000 --> 31:26.000] well the plane just flies itself doesn't it? [31:26.000 --> 31:28.000] you just put it in autopilot [31:28.000 --> 31:30.000] you don't really need to be there as a pilot [31:30.000 --> 31:33.000] and he says well yeah if everything's going well that's fine [31:33.000 --> 31:38.000] you could just have someone with a lot less experience sitting there watching the dials [31:39.000 --> 31:41.000] but as soon as something goes wrong [31:41.000 --> 31:43.000] as soon as there's any kind of emergency situation [31:43.000 --> 31:45.000] you want someone there that knows what they're doing [31:47.000 --> 31:51.000] okay it's not usually such a life or death situation if a database has an emergency [31:51.000 --> 31:54.000] but you want someone on hand that knows what they're doing [31:54.000 --> 31:56.000] to fix that and get things up and running again [31:57.000 --> 32:00.000] so you still need someone there to make the decisions [32:02.000 --> 32:06.000] and actually if each database takes less time to manage [32:06.000 --> 32:10.000] that means that one DBA can manage more databases [32:10.000 --> 32:12.000] well we've seen how much more data there is [32:12.000 --> 32:14.000] and a similar number of DBA's [32:14.000 --> 32:17.000] so that seems to probably work well [32:20.000 --> 32:22.000] what about the junior DBA's? [32:22.000 --> 32:24.000] I've not been able to find any figures [32:24.000 --> 32:27.000] but I wonder whether there are fewer DBA's [32:27.000 --> 32:29.000] fewer junior DBA roles [32:29.000 --> 32:32.000] if automation is replacing a lot of the tasks [32:32.000 --> 32:35.000] the day-to-day tasks that junior DBA's used to do [32:35.000 --> 32:37.000] do we still have roles for them? [32:37.000 --> 32:39.000] and if that's the case [32:39.000 --> 32:42.000] how can somebody become a senior DBA [32:42.000 --> 32:44.000] if they haven't been a junior DBA [32:44.000 --> 32:46.000] does that mean that the career path is changing? [32:46.000 --> 32:50.000] and back to autonomous databases [32:50.000 --> 32:52.000] I mean if we believe the headlines [32:52.000 --> 32:57.000] when Oracle announced autonomous databases back in 2017 [32:57.000 --> 32:59.000] DBA's were soon going to be a thing of the past [32:59.000 --> 33:01.000] there was absolutely no need for them [33:01.000 --> 33:07.000] so even if all databases were moved over to autonomous databases [33:07.000 --> 33:12.000] we've got the day-to-day maintenance and patching is automated [33:12.000 --> 33:17.000] but you still need somebody to design the databases [33:17.000 --> 33:19.000] to do the data modelling [33:19.000 --> 33:22.000] to make sure that there's efficient code accessing the database [33:22.000 --> 33:26.000] and actually if you look at some of the reviews [33:26.000 --> 33:28.000] these are just extracts from a couple of recent ones [33:28.000 --> 33:32.000] apparently it's an extremely complicated thing to put in place [33:32.000 --> 33:35.000] so you still need somebody who's an expert [33:35.000 --> 33:38.000] to be able to put that infrastructure in place in the first place [33:38.000 --> 33:44.000] so fortunately the answer to all of that [33:44.000 --> 33:47.000] was no the DBA role isn't obsolete [33:50.000 --> 33:53.000] so what conclusions can we draw from all of this? [33:53.000 --> 33:57.000] the DBA role has changed since 1998 [33:57.000 --> 33:59.000] and it will continue to change [33:59.000 --> 34:04.000] in response to changes in database technology and methodologies [34:04.000 --> 34:11.000] the DBA of 2023 has similar responsibilities to the DBA of 1998 [34:11.000 --> 34:13.000] but does them in a very different way [34:13.000 --> 34:16.000] and so they're expected to be able to do very different things [34:16.000 --> 34:20.000] relational databases are still going strong [34:20.000 --> 34:24.000] they're still the majority and we're not expecting them to go anywhere [34:24.000 --> 34:28.000] but DBAs will be expected to know about other databases as well [34:30.000 --> 34:33.000] the amount of data is growing exponentially [34:33.000 --> 34:35.000] being managed by a similar number of DBAs [34:35.000 --> 34:39.000] so that suggests that we really do still need those DBAs [34:41.000 --> 34:45.000] and things are moving either to the cloud [34:45.000 --> 34:50.000] or to a cloud like infrastructure on premises [34:54.000 --> 34:57.000] so the main thing for me is don't panic as we've seen [34:57.000 --> 34:59.000] quantity of data is growing [34:59.000 --> 35:02.000] a lot of that data is going to be stored in databases [35:02.000 --> 35:05.000] and that's going to have to be managed somehow [35:05.000 --> 35:07.000] even if things are automated [35:07.000 --> 35:09.000] you want someone on hand in an emergency [35:09.000 --> 35:12.000] you want somebody to design those databases [35:12.000 --> 35:16.000] you want somebody to write and improve the automation code [35:16.000 --> 35:20.000] you want someone to fix the interesting queries [35:20.000 --> 35:23.000] that have been created by ORMs [35:23.000 --> 35:25.000] so it's a role that's evolving [35:25.000 --> 35:27.000] like any other role in IT [35:27.000 --> 35:29.000] like any other role anywhere else [35:29.000 --> 35:31.000] but it's not going away [35:31.000 --> 35:33.000] if you're happy to embrace automation [35:33.000 --> 35:35.000] and to try new things [35:35.000 --> 35:38.000] you've got a long and happy career in front of you as a DBA [35:39.000 --> 35:42.000] and even if you're a stick in the mud old-fashioned DBA [35:42.000 --> 35:44.000] and you don't want to change [35:44.000 --> 35:46.000] well there are going to be enough legacy systems [35:46.000 --> 35:48.000] to keep you in work for a long time to come [35:48.000 --> 35:50.000] so you'll be fine until you retire [35:50.000 --> 36:00.000] that's it [36:04.000 --> 36:06.000] I got through that faster than I expected [36:06.000 --> 36:09.000] so I've got time for questions [36:09.000 --> 36:19.000] so just a note from my side [36:19.000 --> 36:22.000] what I discovered is very interesting for me [36:22.000 --> 36:24.000] is the operator concept in Kubernetes [36:24.000 --> 36:28.000] and instead of using a managed PostgreSQL database [36:28.000 --> 36:29.000] which is expensive [36:29.000 --> 36:32.000] I just use an operator and the operator itself [36:32.000 --> 36:36.000] it doesn't just install a database automatically for me [36:36.000 --> 36:39.000] it also does piggy-bounds [36:39.000 --> 36:41.000] and it does switching from left to right [36:41.000 --> 36:43.000] and I think this will be the future [36:43.000 --> 36:45.000] so a lot of knowledge we have right now [36:45.000 --> 36:47.000] will be moved into an operator [36:47.000 --> 36:50.000] it's more central and it will be much better for us [36:50.000 --> 36:51.000] I agree [36:51.000 --> 36:54.000] if you're going to have a containerized environment [36:54.000 --> 36:57.000] operators are definitely the way to go [36:57.000 --> 36:59.000] the problem that I've seen [36:59.000 --> 37:01.000] and I guess because of the role I do [37:01.000 --> 37:03.000] I see when there are problems [37:03.000 --> 37:04.000] is that people think [37:04.000 --> 37:06.000] that does everything, we don't need DBAs [37:06.000 --> 37:10.000] so you still need that DBA knowledge behind it [37:10.000 --> 37:12.000] definitely [37:12.000 --> 37:15.000] so working in smaller companies [37:15.000 --> 37:18.000] the DBA is basically if you're lucky [37:18.000 --> 37:21.000] the DevOps team and if you're unlucky [37:21.000 --> 37:24.000] the developer team which means [37:24.000 --> 37:26.000] a lot of people don't have the knowledge [37:26.000 --> 37:28.000] or the interest or time [37:28.000 --> 37:32.000] and things are done in a sort of not proper way [37:32.000 --> 37:35.000] so as somebody who is interested [37:35.000 --> 37:37.000] but has a day job as a developer [37:37.000 --> 37:40.000] how can you foster that environment [37:40.000 --> 37:43.000] where developers will start to understand [37:43.000 --> 37:47.000] if they're making inefficient queries [37:47.000 --> 37:49.000] or if they're building the database wrong [37:49.000 --> 37:52.000] and get them to both know and be interested [37:52.000 --> 37:56.000] and also get the company to kind of care about those things [37:56.000 --> 37:59.000] rather than keeping them until it's an emergency [37:59.000 --> 38:02.000] like you said then the plane is crashing down [38:02.000 --> 38:04.000] That's a really good question, yeah [38:04.000 --> 38:07.000] so I wish I had all of the answers to that [38:07.000 --> 38:09.000] because I've been in lots of organizations [38:09.000 --> 38:12.000] as a DBA or helping [38:12.000 --> 38:16.000] and we've had exactly that situation [38:16.000 --> 38:19.000] so I would say one thing is make friends [38:19.000 --> 38:21.000] make sure that all teams [38:21.000 --> 38:24.000] are actually talking to each other [38:24.000 --> 38:28.000] as a DBA I was always kind of trying to [38:28.000 --> 38:31.000] integrate the other teams and actually be useful [38:31.000 --> 38:34.000] I mean obviously sometimes I was that typical grumpy DBA [38:34.000 --> 38:37.000] no you can't do that, no you can't have all these permissions [38:37.000 --> 38:39.000] I was enforcing rules [38:39.000 --> 38:42.000] but I tried really hard to be there [38:42.000 --> 38:45.000] and helpful and show that I could actually [38:45.000 --> 38:47.000] help to make this query run faster [38:47.000 --> 38:49.000] or help to make that better [38:49.000 --> 38:51.000] so I think that's... [38:51.000 --> 38:54.000] and I did training courses for our developers as well [38:54.000 --> 38:57.000] to teach them about the basics of databases [38:57.000 --> 39:00.000] that's great because I found that most developers [39:00.000 --> 39:03.000] are really open to that because they want to learn [39:03.000 --> 39:08.000] as far as people higher up in the organization go [39:08.000 --> 39:11.000] I'm not sure, I don't know what the answer is there [39:11.000 --> 39:14.000] but probably just showing them how it benefits the organization [39:14.000 --> 39:16.000] is the only way to go there [39:19.000 --> 39:21.000] Hello, thank you for your talk [39:21.000 --> 39:26.000] I feel completely, I agree completely with your talk [39:26.000 --> 39:28.000] I'm a DBA as well [39:28.000 --> 39:30.000] I have a couple of questions [39:30.000 --> 39:34.000] First thing is the role of the DBA [39:34.000 --> 39:37.000] is still an old male role [39:37.000 --> 39:41.000] so what can we do for making it more attractive [39:41.000 --> 39:43.000] for the woman? [39:43.000 --> 39:46.000] because it's something very stressful [39:46.000 --> 39:50.000] it's something that is very stressful [39:50.000 --> 39:51.000] another thing [39:51.000 --> 39:53.000] another million dollar question [39:53.000 --> 39:56.000] it's something that is not changed yet [39:56.000 --> 39:57.000] it's still old male [39:57.000 --> 40:01.000] and I don't think it's a good idea to continue this way [40:01.000 --> 40:03.000] I know it's an horrible job [40:03.000 --> 40:06.000] but how can we make it more attractive [40:06.000 --> 40:08.000] and the other thing is [40:08.000 --> 40:11.000] a lot of companies, a lot of my customers [40:11.000 --> 40:13.000] struggle to find DBA [40:13.000 --> 40:16.000] how can they make it more attractive [40:16.000 --> 40:19.000] for getting a permanent DBA [40:19.000 --> 40:22.000] because I give advice of hiring DBA [40:22.000 --> 40:24.000] because it's important to having in-house [40:24.000 --> 40:26.000] but they struggle to find DBA [40:26.000 --> 40:28.000] thank you [40:28.000 --> 40:31.000] The first question I wish I knew the answer to that [40:31.000 --> 40:36.000] I have no idea how to get more women into being DBAs [40:36.000 --> 40:42.000] that's something that's a problem across the whole of the IT community [40:42.000 --> 40:45.000] and something that's not changed since I started 25 years ago [40:45.000 --> 40:47.000] the only things that I do [40:47.000 --> 40:53.000] is I go into part of an organisation that does coding workshops for girls [40:53.000 --> 40:55.000] to try and get people in early [40:55.000 --> 40:59.000] I mean I was so lucky because my dad was a self-taught coder [40:59.000 --> 41:01.000] he loved IT [41:01.000 --> 41:04.000] he built me my first computer and showed me how to play with it [41:04.000 --> 41:06.000] that's what people need, they need to see that it's fun [41:06.000 --> 41:08.000] I mean, well, we think it's fun [41:08.000 --> 41:10.000] I know it's not everybody's idea of fun [41:10.000 --> 41:13.000] I know, my husband still doesn't understand [41:13.000 --> 41:16.000] and I say no, I know but it's my hobby as well [41:16.000 --> 41:18.000] so I think that's it [41:18.000 --> 41:21.000] and that's probably the same as that first question [41:21.000 --> 41:24.000] is that we need to make this look like fun to people [41:24.000 --> 41:28.000] and maybe we need to try and get rid of that image of the one PDBA, I don't know [41:28.000 --> 41:31.000] We need to clean up our ugly aspect [41:31.000 --> 41:34.000] Exactly, more flashing badges [41:40.000 --> 41:42.000] Hello, thank you very much for the talk [41:42.000 --> 41:45.000] I wanted to ask you, I'm a data engineer myself [41:45.000 --> 41:47.000] I'm a data engineer myself [41:47.000 --> 41:52.000] and I feel that it's complicated to get across what we do every day [41:52.000 --> 41:56.000] because we don't do any very visual things [41:56.000 --> 42:01.000] we're not web developers so we can show really nice pages or nice graphics [42:01.000 --> 42:07.000] Do you have any advice to give for somebody who's in the field of databases [42:07.000 --> 42:13.000] and behind the scenes to make their work more noticeable? [42:13.000 --> 42:16.000] Yeah, that's a very good question [42:18.000 --> 42:22.000] I don't know, I suppose it depends which audience you're looking at [42:22.000 --> 42:25.000] are you looking at internally within an organisation [42:25.000 --> 42:30.000] or outside to the general public? [42:30.000 --> 42:34.000] internally but for people who don't know much about it [42:34.000 --> 42:37.000] people who do other things that manage it [42:37.000 --> 42:43.000] Yeah, I think some of it is showing the value that they bring [42:43.000 --> 42:47.000] the problem with a lot of things, I can speak for being a DBA [42:47.000 --> 42:50.000] is that when everything's going right you're not noticed [42:50.000 --> 42:53.000] because what's the DBA doing? [42:53.000 --> 42:55.000] We're not with nothing's going on [42:55.000 --> 42:59.000] we all need to start putting little things in the code [42:59.000 --> 43:02.000] to cause problems every so often [43:02.000 --> 43:08.000] so maybe just try and bring attention to the value that you bring [43:08.000 --> 43:12.000] Claire is probably a very good person to talk to later [43:12.000 --> 43:15.000] because I don't know if you saw Claire's talk earlier [43:15.000 --> 43:18.000] about how to write things in blogs [43:18.000 --> 43:22.000] so that you get your point across clearly and that kind of thing [43:22.000 --> 43:32.000] Claire has some very good ideas on how to communicate