How to analyze free text in a survey using AI? Classification, aggregation, and confirmation procedures

Conclusion: In survey analysis using AI, the answer ID is left, the free text is classified, the denominator of the aggregation is specified, and then a person checks the original text and the numbers. If you draw conclusions based only on the AI ​​summary, you may miss answers on multiple themes or a few important opinions.

What are the steps to analyze free text in a survey using AI?

ProcedureTaskThings to check
1. Decide on the purposeNarrow down to one task you want to improve and one question you want to knowDon't mix things up that you can't judge from the answers
2. Organize the CSVAlign answer IDs, questions, and free text for each line and answerCheck for blank spaces, duplicates, personal information, and garbled characters
3. Define the classificationHumans read some answers and decide the boundaries of the classificationLeave "Other/Unknown"
4. Temporarily classify using AIFor multiple themes, keep the answer ID and put them on separate linesLeave the original text of the basis and the confirmation flag
5. Verify the calculations manuallyCompare the number, percentage, and minority opinions with the original dataAttach aggregation conditions and revision history to the report

Before sending survey responses to AI, please check Q&A regarding input to AI generated by the Personal Information Protection Commission and confirm the purpose of use and data handling on the service side with the person in charge within the company. We will also review whether it is necessary to leave customer names and contact information as is in the CSV.

Q&A regarding input to AI generated by the Personal Information Protection Commissionhttps://www.ppc.go.jp/all_fad_index/fad2-3-3-q1/

How to create a classification table?

When improving services, consider categories that lead to action, such as price, operation, explanation, and response. First, the person in charge reads some of the responses and defines which expressions should be placed in which categories. If only the classification name is used, the judgments of AI and humans tend to deviate.

ClassificationDescriptions to includeDescriptions to not include
Ease of useScreens and procedures are difficult to useDissatisfaction with only explanatory materials
ExplanationGuides and teaching materials are difficult to understandRequest that the function itself does not exist
PriceOpinions about price and fee structureDissatisfaction with response speed
Other/UnknownInsufficient materials for judgmentDo not force assignment to existing classifications

If you change the classification, the previously classified answers will also be reviewed using the same criteria.

How do you count answers for multiple themes?

``The screen is easy to understand, but the price is high'' includes a positive view of the operability and a negative view of the price. Information is lost when you combine it into one emotion. You can use a method that retains the answer ID and leaves the original text of the label and rationale for each theme.

It is possible to add multiple labels to 10 answers, resulting in a total of 14 labels. For example, if there are 6 responses regarding operability, 4 responses regarding price, and 4 responses regarding explanation, the percentage using the 10 responses as the denominator is 60%, 40%, 40%, for a total of 140%. Since this is the result of one person touching on multiple themes, the total should not be adjusted to 100%. Explanation of pie charts from the Statistics Bureau of the Ministry of Internal Affairs and Communications also explains that pie charts should not be used when the total of multiple answers exceeds 100%.

Explanation of pie charts from the Statistics Bureau of the Ministry of Internal Affairs and Communicationshttps://www.stat.go.jp/naruhodo/4_graph/shokyu/pie-chart.html

IndicatorValue of this exampleHow to write it when reporting
Number of free text responses10Denominator. Do not include non-responses
Total number of labels added14 itemsTotal number of items with multiple labels allowed
References to operability6/10 = 60%"60% of responses mentioned operability"

If you want to calculate another ratio of ``6 out of 14 labels have operability,'' write the denominator as 14. Percentage of respondents and label mix are separate metrics.

What should be done before passing CSV to AI?

CSV saves the original file and creates a copy for processing. Based on each answer per line, check for blank spaces, the same answer ID, cells containing line breaks, and model-dependent characters. Even after expanding to multiple lines after classification by AI, the number of answers can be recalculated using the original answer ID. Excel maintenance and calculations are also summarized in Steps to advance Excel data analysis using AI.

Whether you use a free AI tool or a paid tool, the usage plan alone does not determine whether you can enter personal information or confidential information. Use the minimum amount of data necessary in a company-approved environment. When in doubt, check Information that should not be input to generation AI.

How to request classification to AI?

Please organize each answer according to the classification definition.
Please output as Answer ID/Classification/Affirmative/Negative/Neutral/Original text of basis/Confirmation required.
Please separate multiple themes into multiple lines and maintain answer IDs.
Please do not infer reasons or attributes not mentioned in the answer.
If you cannot determine, please mark it as "Unknown" and indicate the reason.

The instructions above are a starting point for classification. Just by erasing proper nouns does not necessarily mean that it is no longer possible to infer the individual. Please check your company policy and approved environment before sending live data.

Where should I check the AI ​​classification results?

First, check for unknowns, multiple themes, strong negatives, and a few serious opinions. Next, we extract some of the common answers and check whether the classification is stable. Even if there are no problems with the classification of the extracted answers, we cannot guarantee that all answers are correct, so decide the scope of confirmation depending on the purpose.

You can leave the answer ID, original text, classification, emotion, rationale, confirmation required, and corrections by people in CSV for classification confirmation. The example lines are hypothetical answers. Even if the same answer ID appears in multiple lines, it will be counted as one when calculating the number of answers. Digital Agency's Notes on Using Generated AI also requires final human confirmation of AI output.

Digital Agency's Notes on Using Generated AIhttps://www.digital.go.jp/policies/genai/agentchat-intermediate

How to report the results of survey analysis?

The opinions of respondents may not be the same as those of customers as a whole. Explanation of the Survey Plan by the Statistics Bureau of the Ministry of Internal Affairs and Communications also explains that bias can occur depending on how survey subjects are selected. The response period, request target, number of responses, and handling of unanswered responses are included, and the trends found by AI are compared with inquiry records as a hypothesis for improvement.

Explanation of the Survey Plan by the Statistics Bureau of the Ministry of Internal Affairs and Communicationshttps://www.stat.go.jp/naruhodo/12_ppdac/plan/plan1.html

The report text can be written as ``A request was made to ○ people, ○ people responded, of which ○ free descriptions were made.Operability was mentioned in ○ comments/free descriptions in ○ cases.Multiple labels allowed'' will allow the reader to follow the population and counting method. Fill in your numbers with real data, don't supplement them with AI guesses.

FAQ

Is the positive/negative ratio enough?

It becomes difficult to know which specific points to improve. Leave the theme and rationale together.

Can minority opinions be excluded?

Don't exclude just the number of cases. We will individually check for safety, major defects, unusable conditions, etc., even if it is only a small number.

Can I use free AI to analyze surveys?

You can try the procedure with public information or fictitious data. When entering actual answers, whether free or paid, please check personal information, customer contracts, usage plan data handling, and internal approvals.

Before consulting ZETTAi

Please organize the questions, answer data, how to handle free text, and the purpose of using the aggregated results. Regarding ZETTAi's AI system development/contracted development, we can consult with you about data confirmation and aggregation methods before analysis. The scope of individual support will be confirmed at the time of consultation.

Articles you may also want to read

Reference materials

The following are published materials from public institutions that are referenced in the text. Enter the document name and link destination separately.

Reference date confirmed: November 14, 2026. The aggregation examples are fictitious values ​​for illustrative purposes.