Move on. Our next presenter is Chalcan who will be telling us about Dissecting corporate culture using generative AI insights from Ads. So, hello, Evan. Then I'm going to talk about whether Alis have edge on information processing. First of, I want to thank the conference organizers for including our work in this great program. My name is Chelsea. I'm a P student from UBC. I'm very excited to discuss the topic on dissecting copy culture using generative AI, insights from Alis reports. This paper is a joint work with K P R and T f, and P and T f also in the audience. So since the culture revolution began a decade ago, researchers have gain a better understanding of corporate culture through values on corporate websites, through service or interviews with executives and employees. And also from nest conference calls, employee reviews and job postings. And also, other research use proxies for the corporate culture such as CEO culture heritage, as well as corporate fors. And however, there remains some unanswered questions. So these insights about corporate culture are from the corporate insiders. Then do capital market participants share the similar view as the corporate insiders? And second, do corporate culture affect stock prices? In R paper, we try to make progress on these questions by applying generative AI to big data. Specifically, R paper asks the following open ended questions from the vintage view of salt equity As. First, what kinds of cultural values prevail in modern corporations? We might think, it's really hard to measure corporate culture at a firm level. And second, what events people and system ship copy culture? There people have document some links, but there is no systematic view about this. Third, how does a cultural value affect different basis outcomes? That we also do implication of our cpory studies, we link the Cporate culture to the stock price formation. We expect to make contributions in the following literature. First, corporate culture literature, our study provides a new insights from the vintage view of the information intermediaries in the capital market equity allys their insights could have facilitated a better understanding of the mechanisms through which corporate culture affect business outcomes with its implications to management. Second, our paper differs from the current research in generative AI. In the sense that we combine the generative AI with some smaller n language models like Bird and use name as reasoning agent to extract the court effect information other than do the prediction task and the classification task. And we also demonstrate its effectiveness in our paper. And the R study also as to the literature or quits, especially to the student that by applying the nice language models to research reports to get insights into information discovery and the interpretation role. Specifically findings suggest that be able to understand intangibles such as copyy culture can improve the information process ability of st quits expected to know about copy culture? So quits are known for their information processing, producing and disseminating role in capital markets. And to do your job well, let acquire in depth information of firms cover for informal channels like Ernest conference course. And there are more studies suggest that the informal channels are more important, such as conducting set visit, have direct connection with management. And prior work also shows that At have in depth non financial information about the firms cover, such as the management quality and corporate innovation. And in this paper to answer our research question, the task is to extract culture related cause effect relation in S reports. To give a sense of the us reports on average reports have ten pages about 17 sentences. Then in this us reports, as we want to get what is the culture type and the case and the effect of the corporate culture. Here are two examples. So let's read this example as R and maybe and what's the information we can extract from this example. So say that ARG is led by a scrap entrepreneurial culture, led by its founder and CEO. Then the culture type in this prom is entrepreneurial culture, and the course of this culture is founders and CEO. Let's look at another example. This is also from the ALS report. So append say, Oh, give us a concrete reason why culture matters to the bottom nine. Front nine employees are handling the inventories, which are one of the biggest asset categories at most retainers. Then the culture type here is people oriented culture, and the consequence of this culture is the bottom line. We apply the generative AI to do task due to its emergent abilities. So the models employed in R studies has about 1,000 times more parameters compared to the traditional bird models. So this enables the model to perform tasks like comprehending and abstracting complex information from A LS reports. And to give more concrete reasons, two elements of the emergent abilities are zero short and few short learns. So this means that the CGT perform tasks on which they have not been explicit in trained on and on which they only have access to very limited. So And this provides a roadmap of applying generative AI to extract information. The highlight here is that we combine the AI tools with its limitations such as cost contact limitations, as well as androcenations and card bias. With modernized L to effectively filter culture related segments and we ask the CGT to extract the following information like cause effect relation, to, and the reason why ask why Alice discussed corporate culture so that we know how CGBT make this reasoning And this figure shows a bird view of our course effect nonag graph. And the wiz and heighte of each entity represents the segments that represent the relevant segments. So we can see the top three courses of corporate culture this is strategy, strategic transformation, and measurement teams. And top three culture types are customer oriented innovation and adaptability. And top three consequences are market share growth and profitability and the employee satisfication. Due to the interests of time, I cannot go to the details, but if we look at the detail, If we look at what specific type of corporate culture, we think these predictions are consistent with our priors. Then we apply these culture measures into empirical studies. So our sample consists about 28,000 firm year observations covering the SMP 1,500 firms in the last 20 years. And our key measures, we use three key measures of corporate culture. The first is culture discussion, which is a domin variable if a culture is discussed in As reports. And the second is number of values, which is the scope of narrow culture discussion. The number of cultural values have been discussed for this firm. And we also ask GT to classify each segment as negative, neutral or positive. And then we have the to measure, which is the average ton of culture related segments in LS. And we also match each report to identity and to get the S characteristics from Capital IQ ABS and also S research output from ABS and us. Summary statistics, about 42.5% of the firm year observations have at least one culture related discussion. On average, two out of cultural values are discussed in As report in the year average ton of cultural discussion is positive. And if we look at the statistics in other way is that 3% of culture discussions are negative. However, there are two concerns. The first concern is that met just repeat what they have heard from the Ernest conference course. The second concern is that As have a lot of cultural discussions in their reports. However, these discussions have low insights. They just copy and paste all the cultural discussions in all the reports. To address this concern, we have two steps. First, to address list just repeat what they have heard from the Art conference call. We sample into firm years with art conference course versus from years without conference course. An surprise me, we find that list pay more attention to culture, especially in firm years without course. And this suggests that list are simply repeating what they have heard from the executives. No, we focus on firm years with list conference course, and we split the sample into the sample in executives have in depth discussion of corporate culture versus the sample result. And we find that Ali are more active in years executives discuss about corporate culture. And this suggests that Ali have unique views of corporate culture. And then we have the firm year analysis to associate some firm characteristics with cultural discussions. So we first find that Ali discuss nets about corporate culture if this firm has institutional ownership and a higher board independence. As discussed more about corporate culture if there are some significant corporate events such as mergers accuzations and management turnovers. And these are consistent with prior literature. And we also construct the culture mesors from the Ernest conference course, and we also get the culture measures by applying from Glasto from the employee's perspective. And we'll show that there are a positive association between executives, employees and Annet discussing corporate culture. So this is new in the literature. And this regression is at the As year level, and the first three columns will use the firm times year fixed effect. And the next three columns will include Is fixed effect. So what we are comparing is that the Is covering the same firm. And we find that list who are female who have more firm and general experience and who are affiliated with n brokers are more likely to discuss corporate culture. And we also find At with a longer horizon and more general experience, more industry coverage have a wider scope of culture discussion. So they suggest that lit might get information from the PR firms when they discuss the focal firms. So, so far, we demonstrate that insights differ from corporate inside view of culture. So we examine the relation between culture and price formation. The key variable of interest charges classification of sentiment in culture related segments. So just to remind the tone is the average tone of culture related segments in As reports in a firm year. This analysis is at the report level, and we have the firm times year fixed effect fixed effect. So what we are comparing among the other reports that are written by the same analyst for the same firm in the same year. And we find that a change of cartre to from negative to neutral to positive result in about 2.7% increase in the probability of upgrading their recommendations, and also 1.4 inquires in target price forecast And for comparison, we also calculate the toll in the reports that excluding this culture related discussions, we find that the rest of the report, and we find that a change in to result in about 7.8% increase in the probability of upgrading recommendations and 5.8% increase in the target price forecasts. Then to examine whether the wide stock market react to the information in the IS reports, we we regret the allotment stock market reaction on the on the ton and we find that a change in ton from negative to neutral results in addition three day abnormal of 30 basis points around the report day. This corresponds to 88 million increase in the market value for average in our sample. And just to conclude by applying generative AI to Minus report, we find that A's views of cporate culture differ from the corporate insiders view. And A's view of coopory culture are directly reflected in their fundamental research output store recommendations and target price, and also indirectly price reactions to the release of reports. And our study also offers new evidence on how Alice view different for different roles corporate culture can play and different factors that corporate culture can impact with its implication for as prices. And paper also highlights the potential of applying generative AI in information extraction task and offers a roadmap of applying generative AI to finance and accounting research. Thank you. I'm looking forward to a discussion. The discussion is Francesco. Great. So let me thank the organizers very much indeed for having me discuss the paper. And also, I'm sorry that I come after the fantastic discussion by way earlier. So I will never be able to, you know, fill the shoes, but I definitely, my first suggestion for the authors is to really think about all that way also said for the previous paper. As you can see, there is a lot of overlap in the aims of the two papers. So, I always like to first summarize the paper with a picture. This time, it's not a picture about the results, but it's a picture about the methodology, which is the one that we also saw in the presentation. So the authors basically are suggesting, let's use CGPT to perform a set of different tasks that normally we would be willing to perform when analyzing text information. And so I will argue also later that, you know, the paper currently the draft spends quite some time to discuss and validate every single step of what we see here in the picture. I think most of that is not necessarily useful in the sense that we can do all that also in other ways. What's really crucial is the very last piece, which is the fact that in this case, Chang PT can allow us to interpret cose effect relationships in the data. So that's really something as we will see later that other methods we have, I believe, as of now, don't really allow us to do as well. So the authors showed that they can measure cause and effects, in this case, in the application of cultural values incorporations and also that this correlates with sort of information and data that is simultaneous, such as analysts own recommendations in the same reports. So let me stress also here, when we look at the economic findings, for example, relative to the earlier paper, we are not trying here to predict or to forecast future performance. So this look at bias issue, potentially, you know, we don't necessarily have to respond on that. So in terms of my comments, then, I will first start with, you know, thinking about the contribution in itself, because as Way was saying before, I think whenever we see these papers on CGPT, the point is always what is really the contribution? What are we learning here? So let me go straight into that. I think so far the draft kind of includes or discusses three potential contributions. The first time is, you know, it's the first time CGP is used as a reasoning agent. So somebody who is not just mechanically learning from the information based on the initial program. It's being used for a set of tasks, chunking, filtering different segments, measuring the tone of different segments of text, and so on and so forth, and then finally to measure those cause effects. Now, as I was alluding to also at the very beginning, I'm not completely sure that all the first list of things is something where I would focus a contribution. So we have been using reasoning agents looking at text since when text existed, it's humans and humans can reason and can provide us some reasoning out of those texts. You know, that's also why, in fact, I mean, you yourself provide us evidence to validate your measures, having also individuals going over them and showing us that the results are quite similar. Analyzing the, chanking filtering. Again, that's things we can do in many ways with sort of other types of even non generative forms of textual analysis. So really, the contribution, I think, methodologically we want to push for the contribution needs to be there. So this relationship between causes and effects in dimensions that otherwise, we can't get very easily. And so that's where I mean, I would push the authors. I mean, if we want to push for that methodological contribution to sort of then provide us some comparison, some assessment of how does using CGPT for that specific role, compared to what is the workhorse of what literature across several fields has been using so far to detect cause effect relationships in the data, which is directed as cyclic graphs. So what's the o directed asyclgraphs, they started actually, you know, in other fields, no economics. But it's very similar to using some kind of text analysis to identify crucial factors that are kind of related to each other. And then for each of them, providing only one possibility of a causal direction that links the two factors. Now, in economics and finance, most recently actually they have been used to try to understand, for example, how different types of agents, think about the relationship between macroeconomic variables. Now, in particular, we have had obviously this situation which extremely high inflation, very Scaroctic inflation after COVID 19, how do experts and individuals households think, for example, about that. So I'm showing you an application of this technique by Peter Andre and Athers I mean, in a recent forthcoming paper in rested. And, you know, I chose these examples, if you have time to read them. Unfortunately, we don't have time to go for them because, you know, we can see in terms of the experts, the first one is really kind of reborn Milton freedman, if you want. We only have inflation because the Fed is printing money. In terms of households, we really see narratives very similar to I guess what we will see through the presidential election soon. The second narrative here is this kind of greed of corporations inflation is due to the fact that corporations are increasing prices for no reason. And the third one is some kind of I don't know, I would suggest maybe Trump campaign story, like it's the fault of President Biden and given that we don't trust him, that's why inflation is going up. So you can see, I mean, how does the the procedure work, indeed it identifies crucial sort of elements themes that are related to each other and imposes the direction of causation goes in one version only. Now, I would like, I mean, potentially, again, if you want to push this contribution, which I believe is important because, of course, we care about cause effect relationships in any economic outcome to do it in that way. Now, in terms of the contribution for the kind of economic content, I mean, if you want. So what is the economics contribution? You know, we are not in an operations conference. So it's not necessarily just about efficiency here. Obviously, the open question in this literature is due values. That's culture effect, economic outcomes. And so far, the research, I mean, has been actually quite lukewarm in that respect. In the sense that, for example, we know that according to even just looking at correlations, stated values by corporations are not correlated with outcomes of those firms, from the work by Griso Sapienza, Zenarz and many others. We know that values recognized by workers do correlate with a bit with outcomes, for example, work by Alex Edmonds and others, and so on. Think, I mean, in that sense, this paper shows us that the values that analysts, external assessors of the firm recognize seem to be correlated at least, you know, with their own views that then percolate into reactions by investors in the stock market. I think the point here to pin down that contribution though really is, if we compare this paper to the rest of the literature, are we really measuring the same thing? So when we talk about corporate values, what are we really measuring here? And is it the same as we had before? And this brings me to my second kind of area of comments. So, are we measuring culture here? Well, you know, of course, if we want to measure something, we need first to define it. And unfortunately, it's very hard to define culture. Now, the working kind of definition that we were using the literature from the original paper by Oan Chapman in 96 is that culture is a set of norms and values that are widely shared and strongly held throughout an organization. Very nice definition, very hard to measure to provide as a measure in practice. And so, for example, when I look at what the authors did in the paper, I think it's very meaningful. Let's actually build on previous research, not necessarily taking off the shelf, the cultural values. So they also add something more, but also something consistent with what previous people did. Now, one of the issues is that the previous literature was looking mostly at information coming from workers reviews of their own company. So, for example, if I think about the cultural value that the authors also have here, which is innovation, what do workers mean when they talk about values related to innovation, where they mean, you know, a culture of kind of leadership that is very clear within the firm, sort of creativity in the ways in which relationship among teams within the firm are managed and so and so forth. Now, what's the issue here is that then when we move to analysts and analysts talk about innovation in the report, often they will talk about actual operational issues. So this company introduced innovation because introduced a new product, for example. There is a technology that this company is introducing in their production function that actually allowed them to increase their sales, for example. So we're really conflating operations and values and what we meant by culture before. So I didn't know couldn't find much about the data, but then I digged into the appendix exhibit, and so I found some plots about like, what are really? What is the data that we have in here? So here, what we're seeing is what is the number of culture related segments in analysts reports that actually entered the sample here across the industries. So I was really intrigued by seeing that actually the vast majority of those are from finance, for whatever reason, which is also an industry normally incorporate finance we don't even like to study. We exclude, as you know, from our sort of analysis because of specifics about their accounting and so and so forth. And there is huge variation across the industries. So the question is, what are the values that are associated with those industries. What are we measure here? So I wanted to focus just for the case of the discussion on the most sort of represented industries, which is basically the most of the sample we have here in the paper. So finance business equipment manufacturing shops. Like in the next picture, what I show you still based on the appendix is, what are those values that are so highly measured within those industries. And so if you look at finance, for example, what really comes to the eye is like this kind of, you know, sanded colored group of values, which is what we call here risk control. So, what are the risk control? Well, it's often the discussion about the banks, and now banks are actually controlling for the risks. And so within finance, a bank that has higher or lower tier one capital, for example, we compare banks that have risk assessments, risk policies, operations that actually lead to lower, likely to face risks. What about the manufacturing business equipment? It's innovation, the dark green one, which, as I mentioned earlier, also captures the fact that those firms are introducing new products, they have new technologies. So in what sense is that culture rather than being operations. And also, when we think about shops, customer orienter and so on and so forth. Now, if I pick the ones that here I would undoubtedly believe are related to cultural values or what we consider cultural values such as integrity, teamwork, people oriented. They're all very, very marginal. So, for example, teamwork is the brown here at the very end. Integrity, which is a value that in the cultural papers has always been mentioned, Edmons and others, GizoZenas and so on so forth, the orange one is barely ever mentioned, so it barely is part of the sample. So what I really suggest here, you know, of course, then looking at observables comparing only firms within the same industry, doesn't necessarily help much here. Because again, we are comparing a bank that has better risk policies and a bank that has worse risk policies, and not recalrat casture. So first of all, I mean, I would ideally like to see the analysis repeated, but only focusing on those cultural values that we can more likely and more plausibly say are not capturing operations or unobserved regarding to product development, and so on and so forth. And also ideal, I mean, providing more information in the appendix or, you know, online regarding exactly what are these phrases, these texts that belong to every single one of those values you're suggesting. So I'm basically almost out of time. The very last thing I wanted to mention, also something, you know, that actually is in the paper, but very, very buried somehow and not discussed very much with was extremely exciting. So, for example, here, the authors, what we are showing in this hit map is what is the frequency of on average analysts discussing cultural values over time 2000-2020 and also across the reports from the beginning of the report until the very end of the report, where less relevant information, we think probably should happen. And so, what's really interesting here is that there is a clear kind of time and state dependent variation. So during periods of crisis like the 2829 crisis, Saddly nobody talks about culture anymore. It seems to become only important in times in which probably, I don't know, there's not much else to say, times in which maybe having a better culture really brings you ahead because everybody else is doing well. You know, for whatever reason, I mean, I think it would be really great to actually potentially dig deeper into something like that, because that's really an economic contribution that would sort of add to whatever this literature has been doing so far, which is usually just one off measures of culture at one point in time. Alright, so thank you very much. I'm already out of time, so thank you very much for the chance to discuss the paper. Thank you, Francisco for very excellent suggestions. Just to all the points are well taken and just to address some points. So I think one question is really on the conceptual framework. One question is about why what the unique view as could have relative to executives and the values on the corporate website because they have shown there is no cornationtwen, values and the stock price. Action, we think this is because they have the value relevant information. And when they produce the As reports, they really go into the website and they also read the employees reviews. They also serve the employee reviews. And they also serve the employees, and then they get most relevant information and in their S report. So that's why we think this is a bit different. And the second is about our mytolgy. What's our age compared to the direct causal reasoning? A depends on our nature of this research question. We ask the CBT to do a very open ended question this is supervised. We do not specify the causes the culture type or the consequences of the culture. We just ask them to summarize and then we do this calization I think but But I think at the beginning point, we cannot directly apply the method you mentioned, but it's definitely a good point to compare our performance relative to their performance since we already get these culture types. And Alice being the thing when they talk about innovation. We actually we really care about. So we did a lot of steps to make sure this segment is really culture related. To do that, first, do some word ambiguity discrimination that we ask we ask CBD to distinguish whether this segment is about organizational culture is about general. The other thing or is about national culture, and we just include organizational culture. And we will think more about the varied the time varied matures as across s reports. Thank you. I think we have time some questions. Thank you, Francisco. Perhaps I could add a couple of points on DAG and conduct analysis. So this is how I perceive work compared with the DAG literature. Work does not really contradict what the DG is doing. But to build a DAG, you need theory and assumptions, which nodes, point to which node, um, in your work, I think we can use GBT to summarize vast amount of evidence written by Analyst in terms of what is the reasonable way to build a DAG. Like, which nodes are confounders, which nodes are colliders. And this kind of prior knowledge, you need those to build a DAG model. So let's see, um, that's how I perceive our contribution. Um, and to your point about starting with an existing taxonomy of different cultural values. We did not do that because it is a deliberate choice, right? Because nobody has studied systematically, how analysts write about culture. So without using CGBT sifting through all the analysts report, nobody would have known that innovation culture is one of the top value that analysts mention. And when analysts explicitly say that this firm has an innovative culture, it's really hard for us to rule it out. So all other points are well taken. Thank you so much. So, I have a question about the source material that you're using. And for people in analyst literature or for people outside, I think it's well known that analysts reports tend to be biased in nature. And because of that, are you, you know, worried that the way they discuss corporate culture could also be tinted. I think, yeah, I think this is a good comments. I think it could happen that they they cater to the management teams. But on one point, we think this could bias against find the ID results because they just do the chip talk or they just have some value irrelevant information. No, maybe they, you know, bias the way they discuss they change the way they discuss the corporate culture to fit their, for example, target price or recommendations, which are biased. Yeah. But in their stock market price reaction, this could suggest that the stock market could recognize the information from the culture. Is that different. First wide culture discussion can affect the fundamental research, and then wide stock market can recognize this information. Comment is not about your particular paper. So I just received the review paper from the previous discussion, Wj wrote a literature review how do stated technology affect corporate governance studies. I write the first probably ten pages. I'm not done yet. But I think that's probably a my understanding of ways, research and her population in my institute is probably some piece everybody would want to read because there she discussed potentials of using new technology and some problems we should be aware of, including how corporations disclose their information when they know that machines are learning to their information disclosure. Thanks. Maybe I have a response to one point why finance sector has, uh, high has a lot of discussion about cop culture. So I met I met Federal Reserve banker a couple of months ago. He told me that New York Federal Reserve hold a banking culture, uh, conference every year. So that's say something they really care about, uh, uh, uh, uh, risk control culture in banking sector. And two, uh, face points, uh, uh, in terms of S, uh, is best or not, we don't really, uh, take on that, uh, because we find that, uh, cultural discussin has no significant, uh, scison with less earnings forecast accuracy. But also, just affect their recommendation and and target price is kind of, uh, uh, sentimental effect. In this stage. We will look further in next stage. Thanks. Yeah. Okay, so thanks for the spirited discussion. Now it's time to discuss during coffee break.
2024 Corporate Governance Symposium - Session 3 (Generative AI and Finance), Paper 2 (March 15, 2024)
From Nicole Rich April 23, 2024
9 plays
9
0 comments
0
You unliked the media.
"Dissecting Corporate Culture Using Generative AI– Insights from Analyst Reports."
In collaboration with the European Corporate Governance Institute (ECGI), the John L. Weinberg Center for Corporate Governance and the Department of Finance at the Lerner College of Business and Economics at the University of Delaware hosted its 2024 annual corporate governance symposium. The winners of the 2024 John L. Weinberg/IRRCi $10,000 Research Paper Award competition were announced during the program.
The papers included in the Symposium collectively addressed new developments and critical issues within the area of corporate governance.
The Symposium also included the presentation by the one of the authors of the paper selected as the winner of ACC/Weinberg Center Carl Liggio Memorial Award.
- Tags
- Department Name
- John L Weinberg Center for Corporate Governance
- Date Established
- March 15, 2024
- Appears In
Link to Media Page
Loading
Add a comment