IA Summit 2015 Main Conference Talk
Topic(s): case studies and user research
With special recognition to Andrew Schall and Caroline Little
During my previous career as an archeologist, I studied people I couldn’t talk to in order to learn how they interacted with the world around them. As a User Experience Analyst, I now study people I can talk to and learn first hand how they interact with their environments.
In this presentation, I will bring my interdisciplinary perspective to life through a case study of the FCC.gov redesign, demonstrating how my experience as an archeologist made this daunting task doable.
In 2014 we began the research process for redesigning the Federal Communication Commission (FCC) website. The project included intensive research and restructuring of the site. We dug through layers of data and content making fascinating discoveries along the way.
The understanding and research that goes into information architecture is not unlike what goes into the study of an 18th century farmstead. Both disciplines use maps and collect artifacts to create a story of people and information. Using the right methods, you can get the job done with more ease than Indiana Jones.
Every step during the FCC project can be reframed through the archaeological process. The parallels can help you prepare for IA research with the mindset of a historian or archeologist, expanding your approach and enhancing your skill.
- Be a historian: Gather historical materials relevant to the site and talk to the locals when you can
- Survey the area: Have a standard method to your research, this helps you determine when you need to go back and investigate an area further
- Dig deeper: Determine where you need to analyze more
About the speaker(s)
Margaret Alldredge is a recovering archaeologist, current UX researcher at Singularis Tech, and IA nerd.
Margaret Alldredge: To teach you a little bit more about archaeology, students, when they are studying archaeology take what’s called a Field School. That’s where they learn all of their basic skills for digging holes, and filling them back up.
By looking at an exposed bit of ground, you can start taking an idea of how people used the land. You can see from a pile of pipe sand, and window glass that a door used to be located in this area. You really get to understand how people use their environments.
After college, I worked at a Cultural Resource Management Firm. What that is…There’s two different basic types of archaeology. There is academic archaeology, which is more with Indiana Jones was as a professor. Then, there is cultural resource management.
What that is in 1966, Historical Preservation Act was passed. In that, it requires that any project that receives federal funding has to have archaeologist on it, which then resulted in cultural resources management firms.
I quickly went from entry level tech in the field to running the lab in just a couple of years. I decided to go back to school because I was really interested in graphic design and how people use their space. I’m still interested in people and how they use the world around them.
I took a class taught by [inaudible 01:18] on user-centered design and that turned into an internship which turn into part time job which turn into full time. I showed up at Spark and I just never left. I said, “You guys have to deal with this.”
One of the projects we ended up working on was…Actually, before I get into that, just to clarify, when I say site and I’m talking about information architecture, I’m talking about a website.
When I say site and I’m talking about archaeology, I’m talking about artifacts that are all over the place, it’s a dirty hole in the ground. Sometimes it really doesn’t look like much. That’s the difference between the two. I’m going to be saying a bunch of different terms that you’re probably not familiar with, but I’ll try and define them as I go through.
At Spark we ended up winning the redesign project for the Federal Communications Commission. I don’t know how familiar most of you are with this, but majority of people are familiar, popular culture wise with this right here, John Oliver, which ended up resulting in the site crashing.
Margaret: When we were getting into doing this project, we had a few goals in mind. One of them was to combine multiple URLs. The FCC has multiple sites that they’re running like micro sites independently of each other. There’s a lot of data that overlaps. What we needed to do is go through and figure out where that was and streamline the web presence. Part of this was shutting down the transition sites.
They had their old site which is transition.fcc.gov, which is still up and live. Then, they have their new site, their redesign site, but people didn’t really fully buy into that when it got switched. A lot of the petitioners, the lawyers, the congress, they were really holding onto the transition site and didn’t want to get rid of it. Our job was to make sure that there was an appropriate structure that they were OK with and could move forward with.
It was also to improve unstructured IA elements, so there were parts where the bureaus and offices started using the site differently than it was intended, because they really didn’t have the structure or opportunity to use it the way that they needed.
Finally, we were working on creating a unified content process. We’re trying to get everyone to agree that it belongs in a museum, at least in a specific location, all get on the same page.
During this process, I discovered that there were a lot of parallels between my previous work as an archaeologist and my current work, working with user experience in IA. Overall, there are a few steps that paralleled really well. The stakeholder, and content audit, areas of interest, unorganized content, untagged pages, and the IA report.
In this portion of the process, historians will go in before the project begin and gather data and materials. They’ll go to historical [inaudible 04:11] and local historians, and landowners, and gather information about the site where there’s going to be any digging done, and they compare information gathered with surrounding sites if there are other sites that exist.
After gathering the documentation, they come up with a plan how the work will proceed. This is very similar to how the stakeholder interviews worked. We went and we spoke to multiple groups of stakeholders and we grouped them by common interest by bureau, and office, and industry background.
We went into in depth interviews to learn what their thoughts were on the site and what they needed. We also gathered materials from them, from their previous transition. We had a list of all their sites, Excel documents with pertinent information, with the URLs and what’s been transferred over, and what hasn’t been transferred over, so we had an idea of the transition versus the current site.
This list was broken down. It was 31 sites, and from that we were able to come up with a plan on how to move forward, to figure out what sites to look into first and know where we need to pay more attention. From this we learned that we need to gather historical materials relevant to the site, and talk to locals when you can.
The average day of a CRM archaeologist is digging holes and filling them back up. Shovel tests are approximately 30 centimeters around, depending on the state regulations, and then you’re digging down until you get to dirt that people haven’t touched. Because archaeology is the study of people, and if people haven’t been involved we don’t need to know. [laughs]
During the historical process, we figured out the probability of there being a site, of there being information there. We have high, medium and low, then we dig holes at consistent intervals according that. If it’s high, we did them really close together. If it’s low, we separated out and we try and work as a team together to figure out if there’s a site there.
When we do this, we document things like the project name, the coordinates of where we found things, the description of the surroundings, the soil type and color, and the presence or absence of artifacts. If we find artifacts, we’ll mark the hole with [inaudible 06:32] tape.
We’ll highlight it that we go back then we will delineate, the cardinals are actually north, south, east and west. We dig holes all around it to really make sure that that is the extent of the site.
In the context of information architecture, when you have a list of a bunch of different sites. When you’re going into analyze the site, you’re highlighting areas where you notice that there’s something you need to go back to. For this we use the site provided by FCC.
We use the documents to figure out how in-depth you need to go into these different parts. What we saw earlier, there were highlights and these things were indicators to us of where we needed to pay more attention, where we were good to go on things, and where we needed a little bit help.
Going forward with the plan, we had our team of people. We divided and conquered. We had a set plan that everyone was on the same page and we all knew what we were going to be collecting in terms of the data so that we could easily come back together as a group and have a really consistent structure to all the data.
For example, some of the things [inaudible 07:39] , the level of the link, and these are two examples of the type of information we were gathering. Some of the more basic sites that was getting where it was in the site, the URL, if it was a redirect from transition or not.
Then we found in some parts we had to go into the backend of Drupal and collect data that way that we could have more data associated with the information that was pulled from the basic site. That included bureau and office information. We knew who to contact if we needed to get more information about that page.
When it was published, to help determine the relevance of that information. Then we also ended up including Google Analytics to help give it an overview of how often that information was being accessed. In that, we highlighted the areas that we were concerned about. We discuss them and we plan solutions moving forward to accommodate for them.
The takeaway here is to survey the area, have a standard method with your research. It helps to determine when you need to go back and investigate an area further. Areas of interest and features.
When you are digging in the ground, I don’t know if you’ve noticed when gardening, different dirt looks different. When you’re taking Field School, you learn fairly quickly to determine extremely subtle differences between soil types and consistencies. Those subtle differences can tell you a lot about the area.
For example, if it’s a little bit sandier in this one spot or if it’s a little bit of a darker brown that was used differently than the soil next to it. It tells you where people have modified their environment, how they’ve used it, and they’re treated differently than the rest of the site.
They’re drawn specifically on a site map. They are collected in more detail and a standard procedure is used to collect that data. Instead of digging it all out at the same time, you dig out half of it and learn what you can from that, then you carefully excavate the other half.
It helps to produce really detailed information. If it’s something that really warrants you to go into even more depth and look closely around it to see if there’s any other information. Things that we use to help supplement that is we use GPS. We’ll use the satellites to help mark where it is on the site.
When we have all that data from the historical research within apps, we can now determine how it links back and how other sites are related. Then use ground-penetrating radar which I equate to Google Analytics here. With the ground-penetrating radar, you go over the area and you head back and you see where there might be something that you should look into.
When you’re using Google Analytics, you can see, “I know a lot of people are going to this one site. A lot of people are going to this other.” You can also see when there’s something you think people should be going to when they’re not. It really gives a lot of data on that.
In the case of the FCC, we identify the encyclopedia section as an area of interest. It was used differently than the rest of the site. All of the pages had a clear home. There were orphan pages. There was inaccurate locations of information. We paid closer attention to this area. We gathered the data from the backend.
For example, we had the bureau and offices attributed to each page and we had the data that was published and what type of page it was, because not all this information was something we had actually found on the front end. We have to go in back.
The supplemental data gathered for point of comparison, we use Google Analytics to figure out, again, what people are looking at, how they’re getting there, and if there’s duplicate information that people are going to one of those pages more than the other. These are some of the top pages.
You can see transition was used a lot. It’s kind of 50-50 if people are going to FCC or to transition. A lot of people when they went to the home page, there’s still a link to the transition site. People were given the opportunity to just bail on the new site. They could just quickly click that button and go back to what they were comfortable with.
We learned that we need to dig deeper, determine where you need to focus in and research and analyze more. Sometimes, when you’re doing this step, you’ll realize that you need to go back a little bit and work to that historian phase, and you need to gather a little bit more information and then talk to more people.
There’s the unorganized content and disturbed soil. Nobody likes that. [laughs] Kind of like snakes. Sites can be occupied at multiple periods of time. Usually, when this is done in a normal way, there’s layers, so there’s stratigraphy. Stratigraphy is basically just a fancy way of saying layers.
At the bottom is the older information, is the older parts of the site. As the site’s occupied for longer, more information is piled on top of that and more artifacts are there.
Sometimes, someone will come in during one of those occupations and just dig everything up, and then it’s just a jumble of things, so something that you thought was a 18th century [inaudible 12:45] , ends up having something from the 1940s, just in Excel in there.
It calls the integrity of the site into question. For the FCC, the encyclopedia section qualified is this. It was used as the catch all. The back end users weren’t sure where content fit into the IS structure. From the transition site, they had a little bit more opportunity to customize where they put their data and how it was stored and presented.
In the case of the news site, they would go to an encyclopedia page, which is just actually a listing of pages alphabetically with very little structure. I didn’t know where to put things. They wanted to be able to put them exactly where they want them. There were multiple types of content. This doesn’t even really get the surface of it, but there were FAQ sheets, there were mixes of headline pages where people were gathering data from other parts of the sites put together.
Every single topic, the FCC covers. I have a couple of examples of those. There’s this white space database administration. This was a really long page. The page navigation links there were jump links. Instead of linking to other pages and having a structure for that, it was just one long page with jump links. This happens multiple times.
This one is an example of one of the archives that they had. They didn’t have the opportunity to create an archive on their site. What they did is they made their own archive pages and tried to work around it that way.
Here, this one, I thought it was pretty impressive for them to do. They created their own tabs, and they have search fields in there. It’s much different than what you would think from the intended use of the encyclopedia page, where I think the original attempt was for it to be a block of text.
They made a whole website within just this one page, because they really wanted to have the customizable feature, but they weren’t given that opportunity. Then there were things where I wasn’t quite sure where they came from or what they went with. This is an example of an HTML color generator, where you can put in the code and generate colors.
I’m not sure what they used this for, because they didn’t have any context. This showed a need for an improved informational structure, to give content home. There needs to be a thought process for accommodating the future content so that you know that something is going to be different in the future, there are going to be new topics. There’s going to be new information. You want for them to be able to put that somewhere where it lives and not just not into a catch all.
There’s un-tagged pages and un-provenience artifacts. What provenience is, it is the original record of something. In art gallery, you have the provenience of the painting. You have the artist that did it, and then the artist sold it to this person, and this person sold it to that person. In archaeology, the provenience is where it was in the ground.
On a regular basis, the property owners will go out and collect artifacts from their end and gather them together. When you show up, you realize that there is a site there somewhere, but because they collected that data and didn’t include where they found it and any other surrounding information about it, you don’t have the context for it.
One example, this occurred in Ohio. A landowner had collected a bunch of projectile points, which is what archaeologists call arrowheads because it’s not always going to go in an arrow, so it’s a projectile point. [laughs]
They collected all of these and hot glued them to a piece of wood and painted them like the American flag. We knew there was a site there, and we had a general idea of the time period, because you could figure out what the different points where, but you can’t get any data from them because sometimes the points will still have proteins on it from whatever animal it was it killed. You can’t really get that data after it’s been spray painted like the flag. [laughs]
In a lot of time, the un-provenience artifacts are just a jumble of different things. They could’ve come from a place that was not disturbed soil. That could’ve been in context, but that’s been removed from that, so it’s now not as useful as it could be. It’s still really cool when you find this artifact, but it doesn’t have the same significance.
In terms of the FCC and the encyclopedia, it was suffering from this feat. Taxonomy was unused and the related content was not always related. If you wanted to find something, you won’t necessarily be able to relocate the materials easily. The information is really useful when you could find it, but because you couldn’t find it, it was useless at the same time. You really want to be able to find information when you’re looking for it.
One of the problems that some of the people that we spoke with during the user interviews was, they had this one page that they really, really liked and they found it once, and they couldn’t find it again. Other people, when they found that page, they bookmarked it. Their browser was basically serving as their IA.
They were creating their own. They were going to the pages they needed, and they were creating bookmarks for all of the sections that they use regularly instead of there being a structure already in place for them.
When we were in this process, we realized that there would be an impact on the content migration. Because there wasn’t a content for it, we couldn’t automatically assume that this page goes to this location, this page goes to this location, so we had to go through and call out each individual bureau and office.
Way back, when I was showing the screen that had he bureau and office information and they’re backend data, we have to go back to that and we married that with the content that they were currently showing on their site so that they could see what they’re showing. They could see what they had. We included the page view, so they could see that it was actually being used over the time period.
We sent this out to the bureaus and offices so that they could see the information that they were trying to provide and have an understanding of why it was difficult to find it and come up with a plan on how to make it easier to find.
In doing this, we had a few different tasks that we set out for ourselves to do. This is just a snapshot of a moment in time during this project. We had the content inventory, a prioritization. We need to finalize the IA still and compare site maps to new ones and assess the different content types. Then define the different templates and finalize the taxonomy.
Figure out exactly what we could automatically pull over versus not. Because that was a big impact on really the future of the development of the site. If you confine this problem early on, you can address it a little bit more fully, so that it’s not as big of a headache later on down the road.
The content needed to be analyzed and categorized to fit into the new structure of the site, once we got being further along in that process. In that, we have to plan for content to have a home, so that people aren’t doing things like what they did with the encyclopedia page.
We want them to have a customizable way of putting their data in. Something that is still structured and organized, but they can customize it for what they need. In this, this is an example of one of the spreadsheets that we were going through to do the analysis on. We’re defining things as belonging in certain overarching categories, so, yes, the Media Bureau has much different information than the Wireless Bureau.
But they’d still have things that are resources. They have things that are guides, they have things that are archiving. They have very similar needs and structures in place. We went through to determine what those overlaying structures were so that we could try and make things as consistent as possible, while giving them as much opportunity to make the site as useful as possible to them.
This took a lot of time sitting in the dark on a couch with a computer, going through and individually figuring out what is what. Sometimes, in the case of not having that taxonomy, you can do something where you search what that page content is and the title is.
We’re able to group in that way, so we’re able to reduce some of the workload. But it was still very manual labor. In the end, both processes end with a report. With the IA report, it’s a very digital artifact that you’re creating. There is folders and spreadsheets and more documents and presentations. There’s all sorts of digital documentation.
Archaeologists are really big on going old-school with things. This stack of papers you see there is from a site that I did analysis on. I think this again was Ohio, and it was a paper. You write it down and then you run it in the computer. Then you print that out. They like having physical documentation of things.
Therefore gathers the materials from start to finish. Everything that was done in that historical phase, the historians gathered all the maps, and all the data from talking to the landowners and all the things from archives. From then on, all the other data is gathered and put together.
In the end, you end up with a report that has, start to finish, what you did. So that when the next person comes along, and someone wants to expand the road even further, and that site has already been excavated, they can go back to it and understand what you did, and be able to move on from there, and be able to give recommendations even based off of your work before putting in work on their own.
The final report for the IA portion of the redesign of FCC was handled in a very similar way. Materials were gathered and organized, and documentation was kept to be able to tell where we stopped, where you’ve moved forward from, include the findings of recommendations. This goes over some of the things that we touched on in that report.
We went over the specific bureau and office content that we talked about, the Drupal back-end information, how we cross-reference things, our process for doing the work. So that people could come back and understand why the heck we did that. Then we had all of the documentation, so the content of it with all the sites and the microsites. The Drupal back-end data for all of FCC, and each of the bureau and offices broken down individually.
The Google Analytics were all of the FCC, but then again, focused just on the bureau and office. Because they’re very structured individually, bureau and offices were their own things. That was one of the issues with the IA of the redesigned site, was that it didn’t provide the means for each bureau and office to take care of their own business. In a way that made sense for each of them, because they all had a different perspective.
It also included the proposed taxonomy list. As I showed before, there’s the site report. This is the example of some of the folder structure, where it was all of the information that didn’t apply to any specific bureau and office. But it was all there and they needed to be there.
Then the information broken down by the specific bureau and office, so they could each go in and take a look and figure out what’s going on. Lesson here is to produce records, provide a framework to help future researchers so that when they come back from the next round of improvements, they know what’s been before and why. They’re not questioning, “Why did this person do that?” They have an understanding for it.
Before and after, this is the site before. In both cases, sites before. These are the sites after. In the case of the FCC, this is the prototype that we came up with. The case of the archaeology, this site here that you see being a bunch of dirt and open, is the Print House site down in southern Maryland.
It’s the site of the first printing press south of New England in the United States, and I worked on that on my field school. They rebuilt a structure on top of it so that people could see what it was before. If you ever do make it to southern Maryland and go to Historic Saint Mary’s City, and check out the St. John’s Site, there’s an intro video. If you see someone in a green sweatshirt with a maroon headband, that’s me.
For a more side-by-side comparison, you can see up top their current organizational structure — the FCC, our work, tools and data, business and licensing and bureaus and offices. A lot of people will navigate the site by bureau and office. In the new design, we have the more overall broad category. About the FCC proceedings and actions, licensing and database, reports and research, news and events and for consumers. Because that was also a big portion of it.
Then we also have the opportunity for them to toggle over and switch to browsing by bureau and office. If you look under bureau and office in the top-right there, you can see the transition.fcc.gov link for people to easily bail out and go to the old site.
The new IA was more task-based. People, when we were doing the interviews, were saying, “I’m just going there to do my work.” It’s a work site, it’s not for the public. It’s for the practitioners, it’s for the lawyers, it’s for the radio specialist people. It’s really serious people that are trying to do some serious work, and they’re checking on proceedings and actions.
They’re trying to figure out what the status of things are. They’re going in to licensing and database, to be able to get licenses and be able to search databases. The reports and the research are important. It’s important for them to access all of these things as overall groups. The previous structure didn’t really give them the same opportunity to access information that way.
Overall, the takeaways. First, be a historian. Gather the historical materials relevant to the site, and talk to the locals when you can. Survey the area. Have a standard method for your research. It helps you determine when you need to go back and investigate an area further.
Dig deeper. Determine where you need to analyze more. Pay close attention to features. Realize when your site has an area that needs to be investigated with more detail, in order to be able to better understand the context. Produce records. Provide a framework to help people understand the work when they go back for the next round of improvements they know has been done before and why.
All of these things, I was thinking of as I was going through and doing the IA research for the FCC. I was always going back and thinking of this process. After writing this, I realized this process can pretty much work on anything you want to go and you want to check out what’s been done before.
You want to look around and figure out, assess your situation. You want to go into more detail on things, and you want to pay close attention to things that might be a little bit of a hiccup. Or might be something a little bit different. Then you want to be able to produce the records afterwards, so that you can go back and see what you didn’t understand why. Others can go back and see that.
That brings me to the end of the presentation. [laughs] Does anyone have any questions or anything?
Audience Member: I’m just wondering how you actually got all these different stakeholders to come together from different departments, and actually collaborate and share their information and find where those commonalities were.
Margaret: We worked with the FCC. They had a core group of people that we worked with, and they helped provide us with the different bureaus and offices to speak with. They helped with the scheduling and getting everything coordinated with that.
Then we conducted the interviews, and people were actually really receptive and wanted to help. Because they wanted a site that they could use. That’s pretty much it.
Audience Member: Now that you’ve put all this together, what is your future iterative? Are you still collecting data to see what you could still improve? Or did you leave behind, beside just the records, a process for future people who’ll be doing rather than just a setup or the maintenance over the long haul?
Margaret: We followed up and done validation testing on things. Me, personally, and this work as a company, we like to do things iteratively. We want to make sure that what we’ve come up with actually works.
I’m not sure in the super-distant future, how they plan on keeping on doing that. But for the time period that we’re involved, we’re making sure that we are going over these things multiple times, with multiple groups of people.
We’re not just testing with internal FCC people, we’re testing with external people. We’re testing with a variety of external people in all the different audience groups. It’s not just the one perspective, you’re getting everyone’s input.
Audience Member: I was wondering about the transition.fcc.gov, and if you guys have a plan to phase it out? Like a plan to, for example, on those pages, to provide context, frame it, some kind of way to make it so that those users learn to have the benefits of the rest of the site.
Or what’s the thought on that? Is it just done?
Margaret: When we did the original user interviews, we had people navigate the current site. A lot of them just went straight over to the transition site, so we had a really good picture of how people were using it.
We’re not just guessing that people are using it this way and getting that data. We know that people are going there and getting that data. Our process was to help make sure that those people who are going to that part would have a data structure in place for them to access that information in the way that they need to.
On, for example, let me just go back. Here, a lot of them were going to do specific task-based things, and they could not find that in the current IA. In the new proposed one, when we went back and we tested that again, people were quickly able to get all the different functions that they were bookmarking previously.
They could go there, get all their work done without having to search around, and do even more work. Because a lot of them were lawyers, and they were charging by the five-minute increments or something like that.
It was really important for them to be able to just quickly go in and do something. The fact that they had to search around, and they had to go to the transition site to find their information and get their work done, was a financial burden on them as well.
Our hopes is that the new version will help make it so that they can get their jobs done quickly. They can be more efficient, they can save clients’ money and spend the clients’ money on something else. Here’s my contact information for any of you that had questions and weren’t able to ask them. Also, I’m going to be around all weekend. I look like this.