Q 765. Today, AI solutions can be created through four distinct approaches:
    •    Using conventional AI models and methods (such as rule-based systems or classical machine learning),
    •    Fine-tuning an existing Large Language Model (LLM) to specialize it for a domain-specific task,
    •    Training a new AI model from scratch using raw data and a custom architecture,
    •    Designing solutions with flow and prompt engineering without retraining the underlying LLM.

Compare and contrast these four approaches. Highlight the key differences, advantages, limitations, and suggest when each method would be most suitable.

 

🏆 The best answer will be selected on the basis of:

• Accuracy and depth of comparison

• Practicality in recommending use cases for each method

• Clarity and thoughtfulness in the explanationVisionary yet practical thinking

 

Note for website visitors -

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Monday or Thursday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://quillbot.com/ai-content-detector. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

Conventional AI models, rule-based systems and classical machine learning are a foundational approach to AI. Used in situations where data is limited or there is well defined rules that exist. These models are characterized by their reliance on predefined rules or algorithms, as opposed to the more data-driven and adaptive nature of modern AI approaches. 

·         Advantages:

They are simple, cost-effective, and can be highly accurate in their domain. 

·         Limitations:

They lack adaptability to new situations, struggle with ambiguity or bias, and are not as flexible as machine learning models. 

·         Use Cases:

Suitable for tasks where the rules are clear and consistent, such as medical diagnosis or fraud detection. 

 

Existing LLM is specialized for a domain specific task, involves adapting the model parameters using a smaller task specific data set and building o its pre-trained knowledge.  This process allows the LLM to learn the domain and improve its performance on specific tasks, effectively transforming a general purpose model into a domain specific expert. 

·         Advantages:

LLMs can automate tasks like data extraction, customer service, and document analysis, saving time and resources. 

·         Limitations:

LLMs can perpetuate biases present in their data, potentially leading to unfair or discriminatory outcomes. 

·         Use Cases:

Chat bots and virtual assistants powered by LLMs can provide 24 Hour and 7 Days a week customer support. 

 

Training a new AI model from scratch with raw data and a custom architecture, is a process involving data preparation, model design, training, evaluation and optimization. This approach allows for tailored solutions to specific problems where existing models may not be suitable. 

·         Advantages:

Developers have complete control over the models architecture, training process, and evaluation metrics, ensuring a high degree of control. 

·         Limitations:

Creating a new model from scratch requires significant investment in resources, expertise, and time in development costs. 

·         Use Cases:

Using custom-built models to analyze unique and sensitive data that pre-trained models cannot access. 

 

Designing solutions with flow and prompt engineering focuses on optimizing LLM responses through carefully crafted prompts. This approach involves using structured inputs, specifying desired outputs, and utilizing techniques like prompt chaining to guide the model toward more accurate and relevant results. 

·         Advantages:

Prompt engineering allows for rapid adaptation of LLMs to new tasks without the need for lengthy and computationally expensive retraining. 

·         Limitations:

Prompt engineering can be challenging for complex, multi-step tasks as it requires careful crafting of instructions. 

·         Use Cases:

Prompt engineering is crucial for creating a natural and engaging chatbot experience. 

Title: Comparing Four AI Approaches Using a GST Audit Use Case

 

In the context of AI implementation, four common approaches are:

 

1. Conventional AI (Rule-Based or Classical ML)
2. Fine-Tuning a Pretrained LLM
3. Training a New Model from Scratch
4. Prompt & Flow Engineering (No Model Training)

To compare these, let’s consider a typical GST audit use case — reconciling GSTR-2A with the purchase register to identify mismatches and generate audit observations.

 

1. Conventional AI

• How it works: Applies predefined rules to flag mismatches in GSTIN, invoice numbers, or tax amounts.
• Pros: Fast, transparent, and works well for structured, repetitive checks.
• Cons: Inflexible; struggles with edge cases or data anomalies.
• Best Use: Standard GST validations across multiple clients.

 

2. Fine-Tuning a Pretrained LLM

• How it works: Trains a model like GPT on past GST audits to understand patterns and draft observations.
• Pros: Produces context-aware, consistent narratives.
• Cons: Requires a curated dataset and moderate computing.
• Best Use: For firms handling high volumes of GST audits with repeatable reporting needs.

3. Training a Model from Scratch

• How it works: Develops a fully custom model using raw GST audit data and tax logic.
• Pros: Fully tailored and scalable.
• Cons: Expensive, time-intensive, and complex to manage.
• Best Use: Large-scale audit platforms or AI products in development.

 

4. Prompt & Flow Engineering

• How it works: Uses a prompt like: “Summarize mismatches between GSTR-2A and purchase register and draft observations.”
• Pros: Fast, flexible, and doesn’t require training.
• Cons: Output can vary; limited handling of complex reconciliation logic.
• Best Use: Quick drafts, low-cost POCs, and small firm automation.

Recommendation: Most Suitable Approach for GST Audits

 

For GST audits involving structured data comparison and observation drafting, the most practical and scalable approach is Fine-Tuning a Pretrained LLM. It strikes the right balance between accuracy, contextual understanding, and automation — especially when supported by historical audit data.

 

For smaller firms or quick deployment, Prompt Engineering can serve as a great starting point.

When researching these approaches, one needs to consider a few factors like complexity, data, performance, and time to develop and deploy. Each approach has its own strength and weaknesses, and one need to be discerning when reviewing the goals of the AI project. Below are some advantages and limitations per approach.

 

Further below is a table with some considerations and a quick view to determine which approach would be suitable for your potential use case.

 

AI Solution Approaches: Comparison and Contrast

 

1. Conventional AI Models and Methods

- Advantages:

    -. This method is generally understood and well established.

    - Uses less CPU power generally.

- Limitations:

    - May not handle complex tasks.

    - Requires lots of specific expertise.

 

2. Fine-Tuning Existing LLMs

- Advantages:

    - Uses existing knowledge and current capabilities.

    - Quicker to deploy.

    - Able to deliver a high performance.

- Limitations:

    - Performance can be inconsistent.

 3. Training a New AI Model from Scratch

-Advantages:

    - Can be set up to specific needs.

    - Can be better in performance.

- Limitations:

    - Resources can be expensive based on CPU power requirements.

    - Needs large datasets.

 

4. Designing Solutions with Flow and Prompt Engineering

- Advantages:

    - To develop this can be done quick and updated too.

    - Can make use of LLMs and avert high costs.

- Limitations:

    - This needs higher expertise in prompt engineering.

 

 

Parameter	Using conventional AI models and methods	Fine-tuning an existing Large Language Model (LLM) to specialize it for a domain-specific task	Training a new AI model from scratch using raw data and a custom architecture	Designing solutions with flow and prompt engineering without retraining the underlying LLM
Key consideration	Rule based	Specialized task	First to market with unique architecture	For general use
Key differences	- Require data in specialized formats - Uses predefined rules and machine learning	- Uses Chat GPT, Deepseek, etc - Require specialized data for fine-tuning	- Require a large amount of backup data - Requires huge time and resources	- Uses ChatGPT, Deepseek or other LLMs for prompt generation - Uses a no-code solution available on the web
Advantage	- Easy to prepare - High accuracy for prediction - Can perform similar and small tasks - Low cost	- Handle all types and formats of data - High performance - Faster development than development from scratch	- No dependency on existing LLMs - Customization as per need - High data security - High scalability/flexibility	- Faster results - Easy to prepare - No dependency on existing LLMs - Very low cost - High scalability/flexibility
Limitations	- Limited scalability/flexibility - Low confidence for unorganised data	- Risk of data leakage - Still requires some historical database - Require coding expertise	- High resources required - Time-consuming - High Risk of failure - High cost	- Highly sensitive - Limited capability
Use cases	- Disease identification based on symptom decision tree - Regression-based demand extrapolation - Selection of raw material as per the decision tree	- Legal contract review - Troubleshooting bots - Domain-specific report generation	- Genome sequence identification - Self-driving car system - AI for music, image and story generation	- Personalized email advertisement - Minutes of the meeting generation - Report fine-tuning

There are four main ways to build AI solutions, each with its pros and cons.

1. Conventional AI (rule-based, ML):
Good for clear, structured problems like fraud detection. Easy to explain but limited with unstructured data or complex tasks.

2. Fine-tuning an LLM:
You take a big pre-trained model and adapt it to your specific domain. Gives great results for focused tasks but needs decent data, skills, and compute.

3. Training from scratch:
Full control, but very expensive and complex. Only makes sense for cutting-edge research or unique problems nobody else is solving.

4. Prompt/flow engineering:
Fastest and cheapest. No retraining—just smart prompts. Great for quick tasks like summarizing or drafting, but not super reliable or customizable.

In short:

• Use conventional AI for structured, classic tasks.

• Fine-tune if you need domain-specific smarts.

• Train from scratch only if you must.

• Prompt engineering for speed and ease without heavy lifting.

onventional AI models and methods (rule-based systems or classical ML):
    •    Best for: Structured problems with clear logic or patterns.
    •    Pros: Interpretable, relatively easy to build and manage.
    •    Cons: Limited adaptability; requires manual feature engineering.
    2.    Fine-tuning a Large Language Model (LLM):
    •    Best for: Domain-specific applications where general LLMs need specialization.
    •    Pros: Tailored performance, improved accuracy in narrow contexts.
    •    Cons: Requires significant computational resources and high-quality domain data.
    3.    Prompt engineering with a general LLM:
    •    Best for: Rapid prototyping or solutions where zero or few-shot learning suffices.
    •    Pros: Fast to deploy, no training needed.
    •    Cons: Limited control, unpredictable outputs in complex tasks.
    4.    Using AutoML platforms:
    •    Best for: Users with limited data science expertise or when speed is essential.
    •    Pros: Easy-to-use, automates model selection and tuning.
    •    Cons: Less transparency, limited to platform capabilities.

 

This table compares a few aspects of the four distinct AI solution building approaches.

 

Aspect	AI Solutions Approach
	Conventional AI	Fine-tuning LLM	Training new AI model	Flow & prompt engineering
Characteristics	Rely on basic techniques such as regression, decision trees, CART or rule-based logic. Requires structured data and programming.	Use a pre-trained LLM to finetune using domain specific dataset to adapt to a particular task. Uses methods such as PEFT, Prompt tuning, RLHF etc.	Build a custom model from scratch using large volumes of raw data without relying on any existing LLM.	Provide structured, contextualized prompt instructions to existing LLMs using few-shot prompting, chain of thought reasoning methods.
Advantages	Good transparency and explainability. Lower cost due to low computing requirements. Provides precise outputs.	Reduced cost as only fine-tuning is required. Benefitted by the LLM vast capabilities. Faster deployment as not required to build.	Complete control over the architecture. Not influenced by existing LLM limitations. Can be customized for highly specialized tasks.	No training or fine-tuning is required. Faster deployment. Cost effective. Flexible adapting to new tasks.
Limitations	Limited scalability options. Cannot function well with unstructured data. Extensive feature engineering.	Requires high quality data for fine-tuning. High computing capability. Carries risk of over-fitting and bias.	Large dataset requirements. Tedious and time-consuming development. High chance of failure.	LLM capabilities limit performance. Inconsistent output to ambiguous queries.
Best suited for	Simple and well-defined tasks. Applications with structured data. Applications with high regulatory compliance.	Domain specific applications. High quality domain specific data. Organizations with non-specialized resources.	Specialized needs not served by existing models. Organizations with specialized resources.	Less critical applications. Faster development and deployment. Scenarios with data limitations.

 

 

1. Using conventional AI models and methods (such as rule-based systems or classical machine learning) - Conventional AI methods includes If-then, decision tree, classification etc
- Easy to implement with limited database and less complex structure;
- There may be a scalability issue when it comes to larger database and handling unstructured data
- Suitable in scenarios like - Spam email filtering, sales prediction etc

 

2. Fine-tuning an existing Large Language Model (LLM) to specialize it for a domain-specific task - 

- Fine tuning is a best fit when it comes to working on specific task/dataset

- This method is best fit instead of working on a model from a scratch

- The disadvantage in using this method is requirement of high skillset and domain knowledge

 

3. Training a new AI model from scratch using raw data and a custom architecture

- Training a new model from scratch will enable a customized approach basis the client requirement

- Similar to fine tuning this method also requires high skillset

 

Both Fine-tuning and Training new AI model is suitable in cases where high precision/accuracy is required.

 

4. Designing solutions with flow and prompt engineering without retraining the underlying LLM

- Taking the benefit of pre trained models, this method is a very quick implementation approach and more adaptable

- Chatbots for customer support or agent queries can be easily implemented using this method.

 

Conventional AI models and Flow & prompt method is the best way for quick and easy implementation scenario whereas fine tuning and training from scratch is suitable for customized high accuracy scenarios

Below is a high-level comparison of the available approaches:

 

Hamid has provided the best answer to this question. Well done.

 

Vinod's answer is also a must read.