Philip Potter

  • Dadding, back at work

    Posted on 06 May 2018

    Previously, on Phil’s dad weeknotes..

    I returned to work from my 14 weeks Shared Parental Leave on 6th March. I stopped doing dad weeknotes after I returned to work, but I thought it was worth another update, because there are things worth writing about in Luke’s development and in my return to work.

    The new routine

    To be honest, it’s been a bit of a bumpy reentry, in terms of working out a new routine that works for my work, my wife Sonia’s work, and Luke’s childcare. On top of that, nursery has been the expected melting pot of pathogens, and we all got a pretty horrible viral infection (I was affected for about 4 weeks, and had 2-3 weeks total time off work as a result). But we’re just about getting settled into a routine that works.

    Luke is in nursery 4 days a week (every day except Wednesday), 9am to 5pm. Every morning, we need to get him out of bed, change his nappy, prepare his breakfast, give him breakfast, get him dressed, and supervise him enough while he plays. At the same time, we have to get ourselves showered, dressed, teeth cleaned, and sometimes shaved, hair washed and dried. Every evening, we need to pick up Luke, take him home, prepare his dinner, give him dinner, give him his bath and brush his teeth, read him stories, and put him to bed.

    My work is flexible enough that I can arrive late or leave early in order to do drop off or pick up, but I still need to put in the hours. So if I need to leave at 4.15pm to get to nursery in time, I need to be at work well before 9. And if I arrive at 9.45pm having done drop off, I stay in til 6pm.

    We have struggled with a particularly gnarly hazard that if we don’t explicitly talk about things, we both end up assuming that Sonia will get involved and pick up the slack. There are at least two reasons for this: one is around overfunctioning versus underfunctioning (and there are definite gender issues here, too), and the other is the practical fact that Sonia works from home, so if I’m doing drop off or pick up, she’s still around to help with breakfast or dinner. This isn’t entirely one-way, either: I find it difficult to leave early in the morning when there’s still breakfast things to clear away, washing up to do,

    We took each week as it came, made plans around who does pick up and who does drop off, and chatted at the end of each week to see how things had gone, what went well and what could be improved.

    Some of the things we have learned after painful experience are:

    • it’s very important to keep talking about things, especially while the routine is still forming
    • we need to have clear expectations about who is lead parent, and when the other parent is around but has work responsibilities and therefore can’t help
    • having dinner options for Luke that can be prepared in a short time with minimal effort takes up-front planning and effort

    This week, Sonia had a business trip to Scotland from Tuesday to Thursday, so I had to do Tuesday pick up, Wednesday looking after Luke all day, and Thursday drop off, all alone. That really helped me build my confidence that I can do the morning and evening routines on my own, and helped me to see all the things Sonia had previously been getting involved in that she maybe didn’t need to.

    Walking

    Luke is now walking! He had been cruising (walking while holding on to the furniture) for some months, then a couple of weeks ago he took his first freestanding steps. There were a few days of taking 3-4 steps and collapsing on his bum, but pretty soon he was taking longer walks, and even squatting down to pick things up and standing back up again. He’s now very confident walking everywhere, and we rarely see him crawl at all any more.

    I had worried that walking would make him even harder to supervise when playing, but it’s actually been more mixed. When he started crawling, we needed to babyproof much more, such as moving things out of his reach. We needed to do a bit more of that for walking, since he can reach a bit higher now.

    But one thing I didn’t anticipate is how taking Luke to the park is much less effort and more reward now that Luke is walking. When he was crawling, he would be much closer to the ground, and he would find things there that he’d put in his mouth. I have horrible memories of trying to fish a cigarette butt out of his mouth that he’d found and immediately put in there. Now he’s walking, he does much less of that, and he can engage with the actual play equipment much more. So where previously I’d have to keep a very close eye, and perhaps take him from one piece of play equipment to another, now I just let him walk where he fancies and slowly stroll round behind him.

    Another thing about walking is that he can now get from one place to another while carrying an object. He showed us this by stealing another boy’s ball when we were at the park and walking around carrying it. So we’ve got him his own ball, and now he loves picking it up, throwing it, and chasing after it. He has such a big grin on his face when he has his ball, and it’s clearly such great exercise for him. He’s not just practising his walking, he’s also practising squatting, balancing while his hands are full, and kicking a ball.

    The loveliest part of this is that he now he can rake around his collection of books, choose one, and bring it to me to read to him. Walking means that he is in control of which books we read, instead of me picking for him. I hadn’t appreciated how walking would change the dynamic of so many of our interactions!

  • Keep database deploys separate

    Posted on 04 April 2018

    Context

    A while ago, I tweeted that you shouldn’t deploy database migrations at the same time as your app code. I thought I’d write something about why I feel this way, and the situations where I feel strongest about this.

    This post has the structure of an Architecture Decision Record, because context matters. I think ADRs are an excellent way to force you to think about the context in which you make a decision, and I’m hoping to use the format to think about the context in which my recommendation applies.

    The team that this recommendation applies most to has the following things going on:

    1. You are building a user-facing application, quite probably a web app for consumption by a browser or mobile native application.
    2. You have enough users that, whenever you deploy, most likely someone is using the app. Therefore, you employ techniques to deploy your app with zero downtime, such as deploying one app server at a time rather than all at once.
    3. You deploy your app multiple times a week (possibly even multiple times a day).
    4. You don’t have maintenance windows (for deployments or otherwise).

    Many teams, having read the Continuous Delivery book, will set up an automated deployment pipeline to ensure there is a clear, repeatable path for incremental code changes to go through various testing stages and end up in production. As part of this, they are probably using a database migration tool (such as liquibase or active record migrations) so that the database schema and data structure can be incrementally updated as the application is worked on. And in the final step of the deployment pipeline, the latest code and migrations are deployed to production in a single deployment stage.

    I anticipate problems with this setup.

    The main issue is that database migrations are inherently riskier types of change than code changes. Problems with database migrations might only show up when you deploy to production, for a number of reasons.

    A migration’s success is contingent on what data existed in the database previously. Development databases are often routinely emptied out between tests. You can trivially add a column with a constraint to an empty table, but a single record will cause the migration to fail. More complex migrations involving rewriting data from one form to another (string to int, or int to date) might fail on corner cases that only show up in “real world” data. More mundanely, a database migration might perform a lot of updates all at once, suddenly filling up a disk.

    Another problem with a production (or production-like) environment, that often doesn’t appear in development environments, is that you will at some point end up with an active user hitting a new version of your app while the old version of the database schema is still active (or vice versa). You are doing deployments in a way that avoids downtime, and whichever way you do it, you will have concurrent active requests hitting old and new code. Development and testing environments often don’t cover this case, because tests are run between deploys rather than during them, and so code is often not tested for schema backwards- or forwards-compatibility. In an environment where developers don’t see the hazards with creating commits which atomically change code and schema, they will do so, causing problems only when people try to use the system during a deploy.

    A final issue is that database changes are much harder to back out than code changes. There’s often no “reverse” migration script available. Even if you have written code to reverse the migration, you might not trust it, especially if there was a problem with the forward migration script. Even if the reverse migration was automatically generated, you might have irrevocably lost data already. (This is not an absolute rule: code changes can also lose data; but their risk profile is lower. It’s easier to lose data on a mass scale with an UPDATE IN statement than in a simple web request handler.)

    Decision

    We will not deploy database changes in the same step as code changes.

    This means that a single deploy pipeline stage should not deploy both code and data without a manual trigger in between the two. Each deploy should be smoke tested, to test that a code change without a data change doesn’t break anything, and similarly a data change without a code change also doesn’t change anything.

    This setup means that every step of the deployment process is verifying the backwards- and forwards-compatibility of your code against your database schema.

    There are a number of ways of achieving this. One simple change you could do is change your deployment steps to deploy code or migrations, based on a parameter in the build. I have seen this work successfully for GOV.UK’s deployments – their deployment step template has a parameter for code or migrations.

    Another way would be to have totally separate deployment pipelines for code and migrations. To the code deployment pipeline, the database becomes an external dependency (like any third-party service). Then database deployments can be explicitly scheduled on a slower cadence, such as weekly. John Allspaw has described this approach at etsy and flickr (see below).

    A very low-tech way to do this would be simply to have a convention that releases can either contain database changes or code changes, but not both, and rely on code review or release management to catch contraventions.

    Note that there are several things that are important to keep, even as we separate out code and database deployments:

    • database schemas should be built from a source-controlled definition, not hand-crafted
    • database schemas should be managed using a migration tool
    • database schemas should be deployed using a deployment pipeline, testing changes first in development and testing environments before being run in production

    Consequences

    Developers will have to adapt to this environment, and understand both that database changes are considered riskier, and that if they want to introduce database changes, they will have to consider how to orchestrate the rollout. A typical scenario would involve three steps: first, release a new app which can support the new schema when it arrives; second, deploy the database migration; third, release a new app that removes the old code for compatibility with the old schema.

    This is what developers should be doing anyway, because otherwise their code will cause problems for users when deployed to production. However in practice they either don’t do this phased rollout, or they sometimes forget. Under the new system, if they don’t write forwards-compatible code changes, their code deployments will fail smoke tests. As a result, there should be fewer instances of failures in production due to mismatched code and schema versions.

    Database changes will become more expensive to push through the pipeline. This is by design, but it does mean that there is a risk of perverse behaviour such as writing fixes in code because it’s easier, when a fix in the database would be more correct.

    This change will not fix everything about database migration riskiness. Trying to update every record in a large table to add a new column will still be slow and fraught.

    A brief survey of other people’s views

    I think it’s worth reviewing other people’s thoughts and experiences here too. Many smart people seem to disagree with me, including the Continuous Delivery authors.

    InfoQ recently published an article about including database migrations in the deployment pipeline, although I actually agree with most of the points they make: database schemas should be defined in source-controlled code, using a migration tool, and deployed as part of a pipeline. The schema should be defined by the application team, not a separate DBA team. The benefits of automation are incontrovertible. The only thing I disagree on is that code and database changes should not be deployed in the same automated step.

    On the flip side, John Allspaw has tweeted: Decoupling schema changes from deployment of the code making use of those changes had huge advantages at Etsy. He has spoken about this in this interview by Jez Humble. People ask him “if you deploy 50 times a day, how do you change the database 50 times a day?” and his answer is “well, you don’t”. At flickr, they had a window every other Tuesday to do database changes.

  • Dad Weeknotes, volume 7

    Posted on 25 February 2018

    Previously, on Phil’s dad weeknotes..

    Planning for return to work

    It feels like I only just started these notes but it will soon be time for them to end (as we know them at least): I only have a week and a half before returning to work. I’ve really enjoyed and valued the time I’ve had caring for Luke and it’s going to be a hard adjustment to go back to full-time work and not see nearly so much of him.

    Sonia and I have had some conversations about the new routine. One practical thing we have had to address is that the nursery doesn’t have enough space during March – Luke will be in for 3 days a week but we wanted 4 – so we need to work out how we will care for Luke in that month. Thankfully, I have plenty of annual leave left so I can take some leave to care for Luke on some of those extra days.

    We have tried to replan Luke’s morning routine. The current routine we’re in means that Sonia often doesn’t start work til around 9.30, because she has to look after Luke while I have my shower, and she has her shower afterwards. This has been okay for the moment but isn’t really sustainable.

    We have also had some difficult conversations about hazards that Sonia anticipates. One is that, with Sonia working from home, it may well fall to her by default to ensure Luke’s dinner is ready for him when he gets home from nursery.

    It’s important for us to keep talking about this. We won’t come up with the perfect plan on day one of me going back to work, but we can reflect on what works and what doesn’t and iterate our routine so that it works for all three of us, without placing unfair burden on anyone.

    Language development

    Luke has definitely shown some signs of developing language. He has already been babbling for some time, but recently he has started to seem more deliberate in his use of sounds. When I go and get him up in the morning, he says “Dad! Dad Dad Dad Dad Daaaad” with a variety of tones and inflections. This is exciting, but my feelings of pride are tempered somewhat because this week he has also said “Dad” when looking at: his mum, his granddad, and a piece of cucumber.

    One thing that is far more clear is that he is starting to recognise things that we say and respond appropriately. During mealtimes, Luke often focuses on his food and forgets to drink his water. When I say “have a drink, Luke”, though, he knows to pick up his cup and (hopefully) have a drink from it, or (all too often) see what sound it makes when he bangs it on the table, or turn it upside down and dribble the water out. We have some books which have a common item on each page (such as “That’s Not My Snowman” which has a mouse on each page, or “This Little Dinosaur” which has a ladybird, or “Wow! Said The Owl” which has an owl), and Luke has learned to go hunting for it when we say “Where’s the mouse, Luke?” or “Where’s the owl, Luke?”.

    Milk ladder

    As mentioned last week, we’ve started on the milk ladder. He’s done very well with the biscuits baked by grandma, and so we moved on to muffins. I baked some carrot and dairy-free cheese muffins (made with a specific amount of milk) and he has been having half a muffin and some biscuits with his breakfast over the past few days. So far, we haven’t seen any definite reaction, so this is really good news.

    Luke continues to drink some coconut milk, though not as much as we would like. He had about 40ml of coconut milk this morning, which isn’t too bad, and is far more than the oat milk we were trying before.

    Supporting mum’s work trip

    Luke’s mum Sonia returned to work at the start of January, and this week she needed to be in St Andrews in Scotland for some work meetings. Since Luke is still breastfeeding, this is somewhat complicated. We considered a few options for how I might look after Luke in London while Sonia went to St Andrews: I could substitute with formula while she was away, or Sonia could express breast milk and store it up in the freezer. However, as mentioned in previous weeks, we haven’t found a dairy-free (or extensively-hydrolysed) formula that Luke will reliably drink. We have also had significant spoilage problems with freezing breast milk – often half of the batches I defrosted would have gone foul in some way.

    As a result, by far the easiest option was for us all to travel to St Andrews. I’m still on Shared Parental Leave, after all, so I have nothing tying me to London, and if we all go then Sonia can continue to breastfeed Luke in the morning and evening and do her work stuff during the day. We can also stay with Sonia’s parents who can provide additional support with cooking, cleaning, and cuddles.

    In other house-husbanding news, I very much enjoyed sewing some buttons on to mine and Sonia’s coats while watching Star Trek Discovery.

    Nursery

    Luke missed his usual full-time days at nursery on Thursday and Friday, but I was able to take him in to nursery for an hour on Monday and Tuesday. There were 3 parents with babies in on Monday, which was slightly more than the nursery had chairs for, so on Tuesday they asked us to come in at staggered times. This meant I brought Luke in in the afternoon instead, which is something I’ve never done before. Normally I take him in for an hour or two between 9 and 11, but on Tuesday it was 2.30 to 3.30.

    It was almost like Luke was a different baby! When I take him in around 9, he usually stays very close to me, and even if he crawls off to explore or to play with a toy he’s spotted, he frequently checks to see I’m still there and still looking at him. But on Tuesday, he was off, crawling quite far away, to places which were almost out of sight, not looking back nearly as frequently.

    It’s hard to know exactly what your own baby is like at nursery when you’re not there, when all you have to go on is how they are when you drop them off or pick them up. I worry about Luke, but this made me worry a little bit less.

    With the work trip to St Andrews, it means Luke is going to have quite a long break from nursery. We’re bracing for re-entry when we take Luke back. It’s probably going to take him some getting used to again.

    Quaker meeting

    On Thursday I took Luke to his first Quaker meeting, in [[http://www.quakerscotland.org/st-andrews][St Andrews]. It was a small meeting, with just 9 people (including Luke) and a dog. Luke behaved himself pretty well. I kept him strapped in his pushchair, and he sat and chatted away to himself. It wasn’t the most silent Quaker meeting I’ve been to, but I think people appreciated Luke’s presence anyway. The dog (called Gunnar) also made his presence known, occasionally walking around and making friends with those present.

    Two of the attenders were students who were attending a meeting for the first time. The welcomer, who had given them a brief introduction to the Quakers and a copy of Advices & Queries beforehand, explained to the students afterward that she hadn’t been to a meeting quite like that before either! 8 adults, a baby and a dog does make for a unique Quaker meeting experience.

  • Dad Weeknotes, volume 6

    Posted on 17 February 2018

    Previously, on Phil’s dad weeknotes..

    Health

    Last week, I wrote:

    Luke’s been having some funny days where he has a much lower appetite than usual. He just eats much smaller portions than we’re used to. But otherwise he’s absolutely fine: just as happy and interested in things as usual, if maybe a little subdued. He’s also been a bit snotty, so we’re hoping it’s just a cold or something.

    It was not just a cold. Luke had diarrhoea and vomiting on Monday morning. We called the doctor and were seen the same day: he checked Luke over and said it was a viral infection, and it should improve with Calpol, rest, and fluids, though we should call again if it didn’t improve. Luke slept a lot that day – I think he had 4 naps in total when he would normally have 2, and the naps themselves were longer.

    On Tuesday, when I got Luke out of bed and changed him, I saw he had a dramatic rash on his tummy and back. So we called the doctor again, and were seen that morning. The doctor took one look and said it was a typical symptom of a viral infection and that, although it looked dramatic, it wasn’t something to worry about. Luke himself didn’t seem terribly bothered by it: he was more active than on Monday and wasn’t itching at his rash.

    On Tuesday night, Luke didn’t sleep well. He was up multiple times, crying in his cot (which we hear loudly through the monitor). A note for non-parents on crying here: babies really do have different cries, and in this case it was the sort of cry which we didn’t feel needed a response from us. Normally he would get himself settled again and go back to sleep. However, this night, it happened several times.

    In the morning, we felt he was well enough to take in to nursery. It was almost exactly 48 hours since his vomiting spell, which is the minimum time to wait before returning him to such an environment. Sonia took him in and I had a bit of a break. However, in the afternoon, the disrupted sleep caught up with me. It was a real struggle to keep an eye on him, to give him the attention and interaction that he wanted and deserved. He was still recovering from his illness and was a little clingy; I just wanted to sit motionless and for him to amuse himself. There was a good moment in the afternoon where I put him down for a nap and went and dozed in bed myself.

    These sorts of moments are the ones I find the hardest: to be an adequate parent when you’re not functioning well yourself. But somehow we made it through the afternoon.

    Nursery

    Luke had his first full days at nursery this week: 9 til 4 on Thursday and 9 til 4.45 on Friday. I thought I would have loads of free time without him, but I spent most of Thursday catching up with laundry, cooking, and cleaning. At least on Friday I managed to get a lunchtime climb in at Vauxwall.

    Luke is settling in more and more. One of the things we have been a little concerned about is how Luke’s sleep schedule doesn’t align with the nursery schedule: Luke generally naps mid-morning and mid-afternoon, but the nursery structure has a single post-lunch nap. But the staff have been very flexible about letting Luke keep his existing schedule. And on Friday, he actually slept just the once, just after lunch.

    The nursery staff are really good about giving a debrief at the end of the day. When I pick him up, they take me through his day: when he slept, how long he slept, how much milk he had, what size portion he ate at lunch, how many nappies he got through, and maybe some highlights of what he did that day. This has been really valuable to understand how well he’s doing and what he gets up to when we’re not around.

    Milk

    Luke has cow’s milk protein allergy, so we currently avoid feeding him dairy at all. Luke has continued to drink coconut milk with variable enthusiasm. He drank 2 oz (about 60ml) on Thursday at nursery, but only tasted it on Friday. He still eats his afternoon coconut yoghurt, so that’s something.

    We are going to start a milk ladder: a structured process of reintroducing dairy to Luke’s diet to see how he reacts. We got the details from the dietician. Step 1 is a biscuit made with a dough that has a teaspoon of milk powder in it, and step 6 is drinking cow’s milk (or formula). The first few steps (biscuit, muffin, pancake) are homemade recipes so that the amount of milk can be carefully controlled. My mum is visiting this weekend and she has kindly offered to bake the biscuits for us.

    Food

    Since recovering from his bug, Luke has had a phenomenal appetite. We did a vegan vegetable curry which is perfect finger food for Luke, and he wolfed it down. At one breakfast he had 1½ weetabix and wanted bread as well. Today, we had a tuna pasta bake and he basically had an adult portion. We wonder if he’s going through a growth spurt of some sort.

    It’s nice that Luke has reached a stage where he is able (and insists!) to feed himself, rather than us having to spoonfeed him. It’s at least one thing that has gotten easier for us.

    Those things your parents told you to do

    The NHS recommends babies to have vitamin supplements. We have not been especially good at making this happen every day, and often gone days at a time without giving him his vitamins. But this week, we have managed to get more into the habit.

    Similarly, you can start brushing your baby’s teeth as soon as they come through, but we have not been good at doing this every day either. When we do manage, Luke wants to do it himself, with all the enthusiasm and inaccuracy one would expect from an 11-month-old. We were given a tip to have two baby toothbrushes: one for Luke to play with, and one to actually clean his teeth. Watch this space for whether this tactic actually works…

  • Dad Weeknotes, volume 5

    Posted on 11 February 2018

    I have successfully finished my fifth week of dadding. You may also want to see volumes 1, 2, 3 and 4.

    Shoes

    We bought Luke’s first pair of shoes this week. He hasn’t needed shoes before because he hasn’t started walking yet. He has been cruising for a while now, but this has been indoors. However, now he has started at nursery, and they encourage outside play, he really needs to be wearing shoes.

    We already had some hand-me-downs that were roughly the right size, which is what he’s been wearing at nursery for the past few weeks, but Luke didn’t look comfortable in them, and he either pulled them off or they fell off by themselves.

    We went to Clark’s and got his feet properly measured and bought a proper pair of crawling/cruising shoes. The difference was immediately apparent: he looked comfortable, didn’t try to take them off, and was happily cruising around the shoe shop bench straight away. I think they’ll make a big difference! It’s just a shame they won’t last long as he’ll grow out of them so soon.

    Nursery

    Luke has been progressing at nursery. Again, I took him in Monday to Wednesday to help him further acclimatize. On Thursday he stayed til 3pm again, and Friday til 4 – the latest he’s ever stayed.

    He’s been coming home covered in paint, pen, sand, all sorts. I think this shows he’s having a great time!

    Health

    This week, Luke had his 12 month health visitor check. It was actually one of the more fun appointments we’ve had. There were no needles or physical manipulation of joints. Luke had his weight, length and head circumference measured, then he was given some toys to play with, and the health visitor observed his reactions. The health visitor was clearly someone who loved her job: she was really good with Luke at keeping him interested in the examination and making everything into a bit of a game for him.

    Luke’s been having some funny days where he has a much lower appetite than usual. He just eats much smaller portions than we’re used to. But otherwise he’s absolutely fine: just as happy and interested in things as usual, if maybe a little subdued. He’s also been a bit snotty, so we’re hoping it’s just a cold or something.

    Milk

    We’ve still not found a great breastmilk substitute for Luke during the day. He didn’t take well to the wysoy formula (see volume 2) and we suspect he may also have a soya allergy. After taking advice from the dietician, we’ve been trying oat milk, but after some initial hopeful attempts, he’s shown less and less interest over time.

    This week, I tried to introduce coconut-based milk instead. Again, there was some early success – he managed around 60ml the first time – but I was struggling to get him to still be interested once the novelty had worn off. I have tried taking a drink from his cup myself, to show him how it’s done and to set an example, with varying success.

    Sonia made a breakthrough, though, when she offered him some milk in an open cup, rather than the sippy cup we had been trying. He seems to like drinking from an open cup, though he needs much closer supervision to avoid spilling it all! We shall see if his interest stays high or if it again wanes as the novelty wears off.

    Social stuff

    Once again, I got out to my Thursday night orchestra. With my new free time during nursery on Thursday and Friday, I managed to meet some friends for coffee and lunch.

    Sonia (aka mum) even managed to have three nights out: Monday, Tuesday, and Thursday. I’m really pleased we’re able to have some sort of “normal” social life.