In March of this year, Bob spoke with Ellen McCarthy, a highly accomplished and distinguished executive whose career started as a junior analyst and ended up reaching the very highest echelons of the US intelligence community.
Ellen’s career began at the office of Naval Intelligence. She then moved to Norfolk and the Atlantic Intelligence Center and would later lead all intelligence activities for the US Coast Guard as their director of intelligence. McCarthy then joined DoD’s office of the undersecretary of defense for intelligence working strategy and human capital management. Later she led the nonprofit public-private partnership INSA (the intelligence and national security alliance), helping make that organization what it is today.
She returned to government service as the Chief Operating Officer of the National Geospatial-Intelligence Agency (NGA), then later led the firm Noblis as its president. Ellen was then appointed the Assistant Secretary of State for Intelligence and Research (INR), where she led an organization famed for the highest quality of analysis in the US IC.
In January, Bob had a conversation with Kathy and Randy Pherson to discuss the third edition of their book: Critical Thinking for Strategic Intelligence. Both Kathy and Randy had successful careers in the US Intelligence Community, where they pioneered new analytical methods and would later help bring those methods to widespread adoption in the community.
The Phersons are also successful business leaders who created companies that build value for others. They provide their context on what a good intelligence program in industry or government looks like, how to avoid cognitive bias and cognitive traps, how to be proactive in data collection and processing, and how to produce intelligence in ways that can be consumed by decision-makers. They also provide insights from the latest cognitive science and do it in a way that can help any analyst in any enterprise improve. We also examine what critical thinking is, and how to teach it.
Kathy reviews what she calls the “Five Habits of the Master Thinker”, which includes:
- Examine your key assumptions
- Consider multiple alternatives
- Look for disconfirming data
- Look for drivers underneath your topic and finder indicators of future action
- Understand the context and how the issue is framed, through framing seek to understand
Randy also worked with Richard (Dick) Heuer on books including Structured Analytic Techniques for Intelligence Analysis.
Topics discussed include leadership styles, intelligence-gathering operations, the history of the OSS and the INR at State, disinformation, critical thinking, mental models, and foresight.
“…I learned that really becoming an expert about the folks that you are supporting or partnering with is more important than being excellent at your tradecraft. So being really customer-focused.”
Gourley: The Atlantic Intelligence Command (AIC) was a great place with a dynamic mission. Did you take away any leadership lessons from that stage of your career?
McCarthy: I was at Suitland [The Office of Naval Intelligence] and I went down to Norfolk for a couple of reasons. But I will tell you a really big reason was in Suitland, you know, we did great work. I mean, Naval intelligence analysts are some of the best in the community. When you think of the sort of who has moved up to senior leadership positions and where they got their start, I think it was the principal lessons we learned as all-source analysts at Navy. But I was writing things, I never really got a sense of who was I writing for, you know, I just did not get that feedback. And so going down to Atlantic Intelligence Command (AIC) and then working for [inaudible] and for the other service elements that were attached to Atlantic Command at the time, it was just very satisfying.
You wrote something and you got immediate feedback, you know it was good, it was bad, it helped. And I loved that. I mean, for me, that was self-actualization was knowing that you were having an impact on a policy or an operation. And you had to be very creative in terms of how you deliver that intelligence. You really were very aligned to your client or your customer or the warfighter. And so, I learned that really becoming an expert about the folks that you are supporting or partnering with is more important than being excellent at your tradecraft. So being really customer-focused. The other lessons I learned: I was given a lot of freedom, you know and those days we were not necessarily told what to write about, but the expectation was that you were going to write messages or write products that were going to have an impact.
“…leadership by being present – setting a strategy, identifying the elements of the strategy, assigning people to actually implement those elements, and then holding them accountable – has been the best model for me…”
Again, you thought about your customer and there were no bad jobs. You could make any job; you could tailor it to the way you wanted it. I was blessed to have had some great bosses and leaders and mentors down there. But there was one boss in particular who I will not mention now, who was not the most inspiring. It was interesting, it was a point in my career. I always thought my boss was smarter than me. And this was the first time I said, I do not know if this guy: I don’t know about this guy. But I will tell you what I did learn was how to continue to operate successfully. Even when you did have a leader who was not necessarily the most inspiring. And that was by really fostering some strong relationships with your colleagues in the office and developing relationships with others in the command and really expanding your network, which was a great way of following the chain of command but also staying inspired and happy.
Gourley: I have seen and worked for the same kind of boss. In fact, from AIC I took away the same kind of leadership lessons. And there are also times when I saw that, well, there is a leader I will not be if I ever get in that position, because you see other people in that same position who have a different approach that led by creating a common vision, aligning people, and encouraging. And they can still be very demanding, but in a way where you understand why you are doing it. And I think people work as hard or harder and deliver more under that kind of environment.
McCarthy: I agree. And also leadership by being present, setting a strategy, you know, identifying the elements of the strategy, assigning people to actually implement those elements, and then holding them accountable has been the best model for me. And then being there, you know? I will never forget Admiral Wilson when he was the J2 it was not unusual for him to walk out at the end of the day and sit down and put his feet up on the desk and just talk to you about how things were going. Or if we send him a message and he liked it, he used to put a smiley face in the corner. I know that sounds so simple, but I tell you, your day could get made if you came back and there was a smiley face written in the corner of a product that you had delivered to him. So those things really mattered. And I will tell you, he was somebody that I definitely tried to model throughout my career.
Gourley: Yes, it is a very good point. And he was a good guy, a great leader. I am sure he still is today. And he would give broad guidance: just do the right thing. for people he trusts, that is his kind of his guidance.
McCarthy: I still stay in contact with him, and I still have followed him. I will never forget I was at a mid-career course in which I got the opportunity to participate. His advice to us was “Work Hard. Play Hard. Sleep Smart.” And I took away from that sleep last, but he really did say sleep smart. And I will tell you, he certainly lived by that, you know, that it is love what you do, but also recognize that what you do also enables you to play golf or to go sailing or to go for a run. And he certainly followed those principles and it worked for him. So, I took the lessons.
“I think the biggest challenge that we face in the next decade is finding out how to know the truth and how to find and get access to it. And what you can trust and not trust. And that must be a long-term effort to build a new culture.”
Gourley: You know, there is a lot of research that has been done on this domain of critical thinking, but I have not seen such a clear and concise definition of what it is, and there is also a lot of talk about what is needed in terms of teaching, even at the kindergarten level of how to think all the way through high school, and then of course, into college teaching critical thinking, and amongst the people I dialogue with, it is almost unanimous. Everybody believes we need more critical thinking at K through 12, which is going to help the nation protect democracy – and help us avoid all these issues with foreign influence and the bias and fake news and other deep fakes – is if we could all just practice more of this critical thinking. I guess what I am saying is I would love a K through 12 edition of your critical thinking for strategic intelligence. It would be a book titled Critical Thinking for Life.
Randy Pherson: It is particularly important as we move into the world of digital disinformation, and there is so much out there. What is nice is that a lot of the high school age people are discriminating readers. They have seen a lot of junk out there and they are more inclined than the old fogies to disbelieve what they get and what they see. But there is a recent article just came out of the survey of all the European countries as to who has done the best to combat fake news and digital disinformation and the answer was the Fins (Finland). The Fins have taken to starting people at K and worked through 12th. And they have very rigorous programs where they create a culture of how to understand good thinking, as opposed to bad thinking.
So, all so they have stopped putting their energy into trying to keep out the bad stuff, and they have replaced it with a positive narrative of “this is what you should be looking for.” This is how to be a critical reader, a critical consumer, and they have been educating the population. And basically, what they have done or the reviews I have been seeing, is that they have inoculated them against a lot of the really the trash that we see all the time because they have an educated population that knows what to look for and what not to look for. And when I’m teaching college classes, I’m now telling the college students, they have a great future in intelligence analysis, except what they’re probably going to be doing is taking what we learned about sources and what we learned about rigor and everything else and applying it to dealing with the world of digital disinformation and trying to keep that under control.
I think the biggest challenge that we face in the next decade is finding out how to know the truth and how to find and get access to it. And what you can trust and not trust. And that must be a long-term effort to build a new culture. And like you are saying, we would love to start it off with another version of the critical thinking book. If there is anybody out there who wants to partner with us, give us, give us an email. We are all set.
“…it is through framing that you understand the totality of your issue, and you can share frames with other people, and you can collaborate as a result.”
Kathy Pherson: And I think one of the things that that is most helpful that we have realized in terms of working on critical thinking kind of exclusively and how it is applied is what are those skills? What are those techniques that are most important? And this is something that Randy, after leaving the UK cabinet office at one point and they was saying: “which of these techniques are most important?” And so, he thought about it on the plane on the way home and ran it by me. And I added another one and came up with what we call the “Five Habits of the Master Thinker.” And so, as I tell classes all I need you to do is think about five things. Number One: learning how to examine your key assumptions.
Number Two: thinking not just about one alternative, but multiple alternatives because there is always more than one. Number Three: look for disconfirming as opposed to confirming data. And this is super important for, you know, instead of investigators, just following a lead, you are always looking for how it is that you can be wrong. This is the scientific method in a world that is not scientific. Number Four: look for the drivers that are underneath your topic so that you can look to the future and use indicators and Number Five – which is my all-time favorite which really should be first: what is the context and how do you frame your issue? Because it is through framing that you understand the totality of your issue, and you can share frames with other people, and you can collaborate as a result.
“I learned that I liked the P&L model. You know that a lot of folks in the government think that making money is a bad thing. Making money is an incredible measure of how you are doing. It is also a measure of the value you are providing.”
Gourley: And then you became President at Noblis?
McCarthy: Well, it was actually Noblis at a subsidiary company called Noblis/NSP (National Security Partners). It was an interesting business model, Bob. So noble itself is a nonprofit. It was it is an offshoot of MITRE, MITRE tech. So very R&D focused, very technology focus, Noblis was the for-profits subsidiary entity. So, any profit we made within folded into noblest continue with its internal research and development programs, and Noblis/NSP was very focused on data analytics and cyber threat intelligence. It was interesting – they were not necessarily the same thing: data analytics and supporting some things that CIA was working on at the time and FBI and then cyber threat intelligence was on the Air Force, NSA side. And so, I got to run a company. It was an interesting time for this company because Noblis had purchased NSP – it had been four separate companies that had been smushed together and then sold.
And so, I got to use some of the things I learned from being the HR lady [at USDI] in terms of how do you bring together four different cultures, how do you motivate people to focus on a smaller pipeline? They had a pipeline that was about as wide as it could be. And so given my time at INSA, it was about focusing on the things, the value that NSP could provide its customers, and really focusing our time on, you know, two or three opportunities versus a hundred opportunities. And it was a P& L, a profit and loss. And I will tell you that I learned that I liked the P and L model. You know that a lot of folks in the government think that making money is a bad thing. Making money is an incredible measure of how you are doing,
It is also a measure of the value you are providing. So, it is not just about making money for money’s sake, it is about providing, providing a service, or providing a resource to somebody who needs it so they can do better. And so, it is so hard to measure value when you are in the government. Although I have some ideas on that. In the private sector, it is much easier. It is you making money. And in this case, it was money that was then getting folded back into Noblis to develop new capabilities. So, I, it was great. It was a great three years.
“Your mental model is simply the way you are organizing the data and the problem that is set before you. I think it something that you can learn how to do better and that you can learn how to do more broadly.”
Gourley: Something else you write about in the book and you mentioned it briefly, Randy, is the cognitive trap. What do you mean by a trap like that?
Randy Pherson: Well, the intuitive traps is the phrase we use. And what we say is we have the high floating phrases, like confirmation bias and anchoring effect, et cetera, et cetera. But when you get down to being an Intel analyst and working every day, they are practitioner’s mistakes, you make like projecting your past case onto the next case. You know, other things were the little mistakes that you make drawing conclusions from the two smallest samples. We came up with about 18 of them as little things that can just do wrong – presuming there is a pattern where there’s not a pattern, assuming that… well, we won’t go through the whole list.
But there are things that we collected as the types of mistakes that you ended up making. And if you can go back with a little checklist, say, did I fall into these traps? then he would go back and correct them. And the idea was just to remind yourself that these are the things that you – by human nature, the way your brain is wired – you are probably gonna end up doing. So go back and check yourself to see if you fell into the trap or not.
Gourley: You also write about mental models in general. But I was wondering do you have a good working definition you can give us on what is a mental model and how does that apply to critical thinking?
Kathy Pherson: Well, to me, it all goes back to that fifth habit of the Master Thinker. It all goes back to the context and the way you organize your reality. So again, a lot of that comes from your experience. It comes from what, you know, it is one of the reasons why people from Steve Jobs to Dick Heuer will write about the importance of reading a lot and expanding your horizons so that you have more knowledge in your head about what alternatives might be. Your mental model is simply the way you are organizing the data and the problem that is set before you. I think it is something that you can learn how to do better and that you can learn how to do more broadly.
“…what is important is the model you bring to the data, not the model that the computer gives to you. You are going to inevitably be influenced by it, but it is your independence of thought and your ability to bring the human knowledge of what makes your model and where it might be different for the machines.”
One of the things that I have learned in a bureaucracy is that that if you want to if you, if you end up being in a situation where you are stuck on a problem, and you are stuck in some sort of a controversy, one of the ways to break the log jam is to expand, expand the model, expand the framework, because you then put it in a larger context. And that changes the dynamics that you are talking about. And so, your ability to expand is what is important. I did a paper. What was that, a couple of years ago? I started to work on creativity and how creativity fits in with critical thinking. You know, you have classes on creative thinking, you have classes on critical thinking. Well, creative thinking is part of critical thinking. They are inextricably linked because you cannot be creative. You cannot imagine the alternatives for your critical thinking unless you are thinking creatively
Randy Pherson: And to jump in here. It is really critical to know what your colleagues are using as their mental models At a conference many years ago, and Philip Tetlock had just started doing his groundbreaking work on foresight analysis. And I was looking at his work and we had lunch and I was saying, well, as people are trying to forecast, what are the mental models they are using to become good forecasters? I said, what I do is I build a model of how things work. And then I fit the data into my model, like political instability or what causes military coups. That is kind of my background.
Other people I know are historians and they work from historical analogy, and they use those models to understand what is going to happen. Or you have the empiricists, who just collect all the data and say, “what does it mean to me? You have the algorithm makers. So, what kind of a process are you using to understand the current to predict the future is really important. And if you don’t understand where people are coming from with the kinds of models they are using, then you can have a lot of disconnects as you are trying to figure out what the future is gonna be.
Kathy Pherson: And the thing that is super important about it is, as Randy said, is to have a mental model. One of the things that really scares me about AI, I remember a [inaudible] talking about “we’re going to live in the data. We are going to live in the model.” Well, what is important is the model you bring to the data, not the model that the computer gives to you. You are going to inevitably be influenced by it, but it is your independence of thought and your ability to bring the human knowledge of what makes your model and where it might be different for the machines. And so, as we write in the book, think about technology, think of AI partner as just another smart colleague down the hall that does not obviate the need for you to do the mental work of doing your own organization and establishing your own models.
“You tell us we punch above our weight, help us out here. We had to articulate where we needed more investment and then ask for it and to move out on it. I think INR, of all the places I worked, is absolutely the best in terms of the quality of analysis and the expertise and their mission to partner with the policymaker.”
Gourley: And Ellen I want to talk about the Department of State and your role there, Assistant Secretary of State for Intelligence and Research [INR]. Now, I think a lot of people watching and listening might not really have heard of this organization. Let me give you my view of this group. And of course, I really want to hear from you since you led it. But, you know, from the early eighties, I had heard about this group and the reputation was here is a cadre of experts at the Department of State that knows something about every country in the world. Maybe that wasn’t true, but you could ask a question of “you name the country.” And they have two people that have studied that for their entire career. And they know the culture, the leadership, the actions, and they have this deep expertise. And through my whole career, this organization had that reputation among the people in the rest of the Intel Community.
Then after 9/11, the reputation grew even more because these guys were just right again and again and again on topics like Iraqi weapons of mass destruction. When I was back in government, I got to see some of their technology infrastructures. I love tech tools and analytical tools, and I go there, and I see that their greatest analytical tool is the brain. They get data into the brain of these smart people and think so maybe it is not the most famous organization in the intelligence community, but it is certainly one of the most highly regarded. But now let me ask you, Ellen, what really is the intelligence and research organization in the Department of State?
McCarthy: Well, Bob, I don’t have much to add to that because you pretty much described everything that INR is. I thought the same thing and I had a chance to be there for the last two years and it is all that and more. I always thought that they chose to be this small boutique capability embedded in State, but I will tell ya. It was our 75th anniversary as an all-source civilian intelligence organization. So, as I was doing the research on sort of the roots of INR, it really was the research and analysis branch from the OSS. INR is the only continuously operating component of the OSS, which disbanded in 1945.
Gourley: The OSS, the famous Office of Strategic Services led by Wild Bill…
McCarthy: …Donovan. Yes. And so, I will tell you that I have spent the last two years reading everything I can about the OSS and Wild Bill Donovan because now it explains everything about who INR is and how it operates. When 1600 analysts went to State, State did not want them because they said things that the policymakers did not want to hear. And because they viewed themselves the diplomats to be, to be truly the elite, the educated, the ones who knew, but these analysts were so good. I mean, they literally were so studied and so expert, and they would come in and tell the policymaker things they did not want to hear. And so, a group of 1600 analysts in 1945 ended up at 300 when I started and I am sitting here thinking, well, how did that work?
You know, why is it, there were over 75 years. Their budget has gotten smaller. The number of people had gotten less, but they are still delivering And they still are just really good. And, and so what I learned was that there really is, and you, you had mentioned that what motivates the Ellens and the Bobs is delivering intelligence to a decision-maker. So, what motivates the INR is delivering intelligence to the policymaker and they are co-located with them in the building. So they will do anything they can to, to make sure that the policymaker, the Secretary of State, the Unders, and the assistants and the desk officers get their views on things. The problem is that they are not as integrated as they were in the past because the world has gotten so much more complex. And you talked about the architecture at state, which is not particularly impressive.
And so, they were still delivering intelligence and leather-bound briefing books every morning, just like you and I did it back in the eighties. So, it was as if time had stopped. And so, INR was small and boutique, not by choice, but just because it needed someone with some background in the intelligence business to say, wait a minute. You know, there are things we can do to help you, there are capabilities we can bring in, let us go ask, you know, we have got this office of the DNI, let us go to the office of the DNI and say, “DNI, we need some help here.” I mean we are the number one producer of the President’s Daily Brief (PDF) per capita. You tell us we punch above our weight, help us out here. We had to articulate where we needed more investment and then ask for it and to move out on it.
I think INR, of all the places I worked, is absolutely the best in terms of the quality of analysis and the expertise and their mission to partner with the policymaker. But they are the most hugely under-resourced organization in the IC. And I am not suggesting that they become another CIA or DIA, their sister, all source organizations of 15,000 people, but they do need a few more people and a few more resources.
Watch or Listen to the Full Interviews:
Kathy and Randy Pherson, Authors of Critical Thinking for Strategic Intelligence
Related Resources:
The Pherson’s Book: Critical Thinking for Strategic Intelligence Third Edition
A Practitioner’s View of Corporate Intelligence: Organizations in competitive environments should continually look for ways to gain an advantage over their competitors. The ability of a business to learn and translate that learning into action, at speeds faster than others, is one of the most important competitive advantages you can have. This fact of business life is why the model of success in Air to Air combat articulated by former Air Force fighter pilot John Boyd, the Observe – Orient – Decide – Act (OODA) decision loop, is so relevant in business decision-making today.
Useful Standards For Corporate Intelligence: Discusses standards in intelligence, a topic that can improve the quality of all corporate intelligence efforts and do so while reducing ambiguity in the information used to drive decisions and enhancing the ability of corporations to defend their most critical information.
Optimizing Corporate Intelligence: Actionable recommendation on ways to optimize a corporate intelligence effort. It is based on a career serving large-scale analytical efforts in the US Intelligence Community and in applying principles of intelligence in corporate America.
An Executive’s Guide To Cognitive Bias in Decision Making: Cognitive Bias and the errors in judgment they produce are seen in every aspect of human decision-making, including in the business world. Companies that have a better understanding of these cognitive biases can optimize decision-making at all levels of the organization, leading to better performance in the market. Companies that ignore the impact these biases have on corporate decision-making put themselves at unnecessary risk.
Other recent OODAcast thematic posts
Richer and Becker on Domestic Terrorism, Cyber, China, Iran, Russia, and Decision-Making
Omand and Medina on Disinformation, Cognitive Bias, Cognitive Traps and Decision-making
OODAcast 9/11 Perspectives
Decision-Making Inside the CIA Counterterrorism Center Before, During, and After 9/11
A CIA Officer and Delta Force Operator Share Perspectives on 9/11
