Derek Willis -

Teaching News Apps with Codespaces

How to spend less time on setup and level the playing field

Teaching Journalism with ChatGPT

Why I want students to enagage with AI services

What is Happening in Morgantown?

How WVU, GitHub Actions & Datasette Produce Good Questions

Welcome Back

A new beginning, but not entirely.

The More Important Problem

The first indication that I did not approach this NICAR-L improvement project in the right way should have been when I posted on GitHub a CSV file with metadata from 16 years’ of posts to the listserv. Not long after, I got…

The NICAR-L Improvement Project

If you’re coming to the 2019 CAR Conference in March, chances are you’re a member of the NICAR-L listserv. The only session idea I pitched this year was on how to…

The Best Training

I’ve been asked by students what the best training is to be a journalist. My position on this is very clear: waiting tables at a honeymoon resort is the best training.

Academy Fight Song, Part 2

It begins, as so many things do these days, with a tweet.

Government and Civic Tech

Looking at what has happened to…

My Favorite Things

As first reported on the tweets, I have a new job, at ProPublica, where I’ll be working on news applications, investigations and other incredible stuff with the News Apps team…

Civic Data and Journalism

A solid foundation of publicly available, consistent civic data -…

We’re All Publishers Now

Last weekend I had myself a proper Twitter rant.

Lightning Strikes

On November 19, 2009, Jaimi Dowdell of Investigative Reporters & Editors sent an email to more than a dozen of us asking about some ideas for advanced sessions for the 2010 CAR conference…

How It Starts

Tomorrow is Aron Pilhofer’s last day at The New York Times.

Data Journalism, Student Media Edition

I had the privilege of speaking to students (and some faculty) at Duke University on Monday, and it was inspiring to see so many people come out to listen to a very geeky talk, to say nothing of the speaker. Afterwards, several students came up to ask…

The Natives Aren’t Restless Enough

A couple of points to start with, in the hopes of not wasting readers’ time and preparing for some reactions:

Teaching Hospitals, Journalism Education and a Hatchet Job

Donica Mensing and David Ryfe from the Reynolds School of Journalism at the University of Nevada, Reno, have published a paper that attempts to argue that recent…

Why Develop in the Newsroom?

Nearly a decade ago, I sat…

What Good is Dat?

Max Ogden, a one-man band of interesting civic (and other) coding, has a new project called dat. Dat is “a new initiative that seeks to increase the traction of the open…

Lessons from ‘Data-Crunched Democracy’

Last Friday I traveled to Philadelphia for Data-Crunched Democracy, a conference drawing together political consultants, data analysis and targeting professionals, academics and journalists to discuss the impacts of “big data” on…

What They Say About Us

As journalism buzzwords go these days, analytics has a lot going for it. The term is broad enough to encompass a wide range of ideas, ranging from the “dark side” where the application of analytics leads to bad and banal writing to the use of…

Academy Fight Song

What’s that I hear
The sound of marching feet
It has a strange allure
Has a strange allure
Mission of Burma, “Academy Fight Song”

The Itemizer

Here’s a little secret about me: I love campaign finance data.

Steve Coll

I have a ton of respect for Steve Coll, who was named dean of the Columbia…

Mobile Apps - Where the data lives

At first glance, I wasn’t sure how useful a mobile app for presidential documents would be. After all, it’s not too…

Magic Removal

Few may remember this, but Aaron Swartz kickstarted Django’s magic-removal effort years ago. Django is 1000x better thanks to his feedback. — Adrian Holovaty (@adrianholovaty) January 12…

On Aaron Swartz

After trading some emails with Aaron Swartz in 2004 and 2005 about building a congressional…

The Data-Driven Congressional Reporter

Washington is full of reporters who excel at finding and building sources, or at knowing which documents to look for and when.…

On Vengeance

I was 23 years old when I saw someone die, and my first thought about it is that I don’t recommend the experience.

Thoughts on NewsFoo

In NewsFoo style, a brief introduction to this post:

How I Got Here

I meant to finish this by Thanksgiving, as a way of acknowledging the key role that female mentors have played in my career. Better late than never.

What I’ve Been Up To Lately

A link dump of election-related work that I’ve contributed to as part of The New York Times’ coverage of the final months of the campaign. In every instance these involved…

Congressional Data on GitHub - A Way Forward

Two years ago, there was a round of blog posts touched off by Clay Johnson that asked, “Why shouldn’t there be a GitHub for data?” My own view at the time was that availability of the data wasn’t as much an issue as smart usage…

Share Your Knowledge

Unlike my friends at The Chicago Tribune, I have no logo to display, but my message here is as important as “Show Your Work”: Share Your Knowledge. In fact, it’s hard to do one…

Finding New Money

My colleague…

Lost in the Weeds

The indefatigable Alex Howard posted a link today about a draft academic paper on open source and journalism by Nikki Usher of George Washington University and Seth Lewis of the University of Minnesota-Twin Cities. Alex’s…

Our Mark Knoller Problem

My colleagues at The Times (and other folks I know who cover the White House) tell me that Mark Knoller, the CBS Radio reporter who reports on the president, is a genuinely nice man and someone who has always been extraordinarily generous about sharing what he knows with…

On Legislative Data Transparency

This week I was honored to speak at at the Legislative Data and Transparency Conference put on by the Committee on House Administration. If you’re so inclined, the videos of the presentations are online at the conference site, although I must warn you that they contain heavy doses of XML references and…

What We Don’t Know About Elections

If you happened to be at the recent Online News Association conference in Boston and happened to attend the session on covering the 2012 elections, then a good bit of this will be repetitive. Since there wasn’t a ton of time to expand on what I said, and I don’t want to leave the impression that I’m critical of all…

In Defense of Building Tools

My first job in Web development was as a member of washingtonpost.com’s “Tools Team.” I was, in title if not in practice, a Tool.

Why Teach SQL?

There was an interesting discussion on the NICAR-L…

Interviewing Data

To my mother’s regret, I was never the literature lover she is. And I am not remotely the writer I might have been expected to be, given that my parents both taught English…

What APIs Mean for Data Journalists

Anthony DeBarros of USA Today and I talked about APIs at this year’s CAR conference in Raleigh. We got a…

Why Students Should Come to the CAR Conference

Update: the student price for the conference, $100, does not include IRE membership. That’s $25. Both are bargains.

Hard Problems

I wasn’t going to respond to Ellen Miller’s comments on my previous post, mostly because I thought I had said what I wanted to. But now that O’Reilly has picked up on things, I figure it might be worth one last attempt on my part. Your experience, of course, may argue against…

How Far We’ve Come

There’s been a bit of discussion lately in the open…

A GitHub for Data?

How APIs Help the Newsroom

As nice as it is to get praised for the civic-mindedness of your work, the not-so-secret secret about APIs at The Times is that we’re the biggest…

Using the NYT Congress API with … Excel?

It’s true that Excel has been a decreasing part of my toolkit for several years now, and that I never quite had the…

A Gentle Introduction to Google App Engine

As part of our roll-out of version 3 of the NYT Congress API, I was tasked with coming up…

Lightning Talks at NICAR

This year’s computer-assisted reporting conference in Phoenix has a couple of new sessions on the schedule. One of them is…

The Gift of Data

One of the more challenging and interesting projects at work lately has been the work we’ve done on the Toxic Waters series by Charles Duhigg. Since the stories have explored water quality throughout the United States, the web component accompanying some of the stories have been national…

The Future of IRE Training

Anyone in journalism who knows me knows how much of a debt I owe to an organization called Investigative Reporters and Editors. Sure, I liked playing with data before I found out about IRE, but the knowledge and support that I’ve received from IRE training, conferences and members has…

A Question of Emphasis

The job cuts at the Washington Post on Friday have produced a round of comments, broadly summed up by Steve Yelvington earlier today. They certainly begged the question that occurred to me as a former employee of both the Post and WPNI, its soon-to-be…

Buying Into Computational Journalism

The intriguing title of a recent report from scholars at Duke is “Accountability Through Algorithm: Developing the Field of Computational Journalism”. Semi-related to CAR, Computational Journalism is defined as “the combination of algorithms, data, and knowledge from the social sciences to supplement the accountability function of…

The FEC’s Disclosure Data Catalog

The good folks at the Federal Election Commission launched a disclosure data catalog recently, continuing the federal government data catalog trend. And while there are few (if any) people better at explaining campaign finance data than…

One Way to Encourage Innovation

Innovation. We’re told over and over (often by people who don’t actually do much more than talk, but that’s another story) that our industry needs it. So, you ask, how I can get me some of that…

The Fundamental Training Need

It’s good to see recent writings on the importance of training and skill development for journalists.

The Case Against Teaching Access

I’ve been at the Medill School of Journalism at Northwestern University since last week, talking to faculty members about using data management and analysis tools (spreadsheets, databases, mapping) in their courses.…

No, Really, Show Us The Data

When it first appeared I was really excited to see Show Us The Data, which gave visitors a chance to list and vote for their “Most Requested Documents” that should be more readily available from the federal government. Sure enough, there…

Represent and GeoDjango

For more details on Represent, see our post on the NYT’s Open blog.

White House Beat Feature Request

Ok, I love the fact that CBS Radio’s Mark Knoller keeps such good tabs on presidential travel, but can somebody please come up with a backup plan in case, heaven forbid, Knoller gets hit by a bus or something? Is this too much to ask of WH reporters? This is the…

The Difference

One of the things I try to stress to students in my computer-assisted reporting class…

The Birth of Quadruplets, or Understanding the Process

My friend Dave Gulliver had a fascinating piece in his paper on Sunday about the birth of quadruplets in a Sarasota hospital. It’s a great story, but what makes it greater is that it was written by somebody with a certain amount of expertise on the subject of difficult premature multiple births.…

Caspio’s Lessons

Been awhile since I wrote about Caspio, and…

The Future of News Libraries

At the recently-completed SLA conference in Seattle, Nora Paul led a session on the “future of news libraries” that asked the attendees to imagine 2012, when librarians (or news researchers, or whatever you want to call them) are…

On Bomb-Throwing

Note to visitors coming via Jay Rosen’s Twitter feed: Nowhere here do I say that…

Of the Web vs. On the Web

News organizations spend a lot of time talking about what they’re doing “on the Web,” but there’s another phrase that’s more important from a long-term perspective: “of the…

Innovation Belongs in the Newsroom

There’s quite a bit of useful discussion about the recent big moves in the social networking space and what it means, if anything, for the news industry. Ste…

The Times

I originally tried to write something about this with the weightiness I felt it deserves, but it turned out to…

Teaching Data on the Web

Matt’s advice (the latest in the series kicked off by Paul Bradshaw) is excellent: “Learn how to put data on the web.”…

On Trials, Software and Otherwise

So in response to several commenters on my previous post, I went to caspio.com to see about a free 14-day trial in order to test things out. Then I read the Terms of Service, which contains this sentence: “In addition, you may…

Outsourcing Database Development, or the Caspio Issue

Updated: Caspio’s David Milliron responds in the comments.

Django, iCal and vObject

For one of our Django applications at work we received a request to add iCal feeds to accompany the RSS feeds available for each candidate’s page (example here). I first thought about doing this using a…

The Original (and Future?) Facebook

So the people who run newspapers are probably looking at Facebook, which is enjoying traffic…

The New Competition

Back when we launched the Congress Votes Database in late 2005, it had only a few contemporaries, including the excellent GovTrack. Now the field is getting pretty crowded, and that’s a good thing for readers interested…

Finding Enterprise Reporting

During our train ride this morning a colleague and I talked about a common problem that has some real implications: the difficulty in finding enterprise and public service…

Shoot the Google

Pieces like the one Neil Henry, formerly of the Washington Post, wrote in today’s San Francisco Chronicle, annoy me to no end. They are written by passionate, intelligent people who have become so disheartened by challenges to their beloved institutions that they feel the need…

Why The Web

Friday was my last day as an employee of The Washington Post newspaper.…

Congressional Vote Database

Xpdf on the Mac

Last year I wrote a piece for Uplink on using Xpdf to convert PDF documents into text…

The Long Bet

It’s Dave Winer vs. the New York Times in a bet over, well, it’s not exactly clear. Dave says it’s about “which will be authoritative,” although that’s not what the question itself says - or what the answer to the question will reveal. In fact, I’m quite…