In This Blog
- Why The AI Sweep?
- Enter Azure AI Foundry
- Getting Started With Azure AI Foundry
- Closing Thoughts
- Frequently Asked Questions
AI, or rather GenAI, is on the top of everyone's news feeds from the medical industry to the automotive industry to the tech industry and everything in between, but what does it all mean? Is the hype just hype or is there an actual use case for engineering teams?
In this blog post, the question on one method for how you can use AI in Azure will be answered with Azure AI Foundry.
Why The AI Sweep?
In the 1950ʼs, roughly ten (10) or so years before Software Engineering came out, AI was first introduced by Claude Shannon. Back then, and until about 2022, AI was out of reach for many organizations and individuals from a consumer perspective. It felt like a mythical phenomenon that you only saw in movies about hackers. Then, OpenAI had just the right tools to showcase in public, but that happened 7 years after OpenAI was created.
AI, or the more known method, Generative AI (GenAI) gave individuals and organizations the ability to begin implemented prompt-based engineering. Ask a question, get a response, and hope that itʼs right. The popularity has now reached just about every industry from healthcare to real estate to construction and especially, software/tech.
In short, it was the right place at the right time with a good amount of marketing that reached the majority of society in a way that they enjoyed.
As with all new tech and inventions that get past “the bubbleˮ and into the hands of just about everyone, engineers and leaders are trying to find a way to best implement these rapidly changing technologies, but itʼs expensive to train Models and run them for chatbot features, coding, and help with automating processes.
Thatʼs where what some are called “SaaS-based AIˮ solutions are coming into play. Letʼs dive into what they do.
Enter Azure AI Foundry
Tools like Azure AI Foundry and AWS Bedrock give engineers the ability to:
1. Use existing Models.
2. Fine-tune the Models on infrastructure that isnʼt theirs, which is important because itʼs very expensive to do so.
3. Use methods like Retrieval-Augmented Generation RAG to train the Models on structured and unstructured data (like a link to documentation) for the Model to do exactly what you need it to do with the proper information.
4. Implement AI Agents to perform particular tasks and actions for you.
SaaS based AI tools, whether itʼs chatbots like ChatGPT or infrastructure solutions like Azure AI Foundry has made AI accessible to the general public without having to spend millions or billions of dollars setting up data centers that have enough power to run and train the Models.
Azure AI Foundry gives you the ability to use existing Models and train those Models for specific use cases that your business needs. That could be anything from training an AI Agent for you to perform a specific action or building an AI pair programmer to help with coding projects to provide the best performance optimization and efficiency for the code thatʼs being written and deployed. Itʼs really a, as already mention, a SaaS based solution for using AI instead of hosting Models yourself. A great thing is it ties into a lot of services that are already in use including VS Code, GitHub, AKS, and many other popular services that are adopting AI.
Getting Started With Azure AI Foundry
Now that you know a bit about the history behind AI, where the market sees it going, and what Azure AI Foundry is, letʼs learn how to deploy Azure AI Foundry in Azure.
1. Log into Azure and in the Azure search bar, type foundry. Youʼll see the Azure AI Foundry service.
2. Click the blue + Create button, and youʼll see two options Project and Hub.
A Project is where team collaboration is vs just having a standalone project. You can share and build multiple AI services inside of a project. A Project exists inside of a Hub. A Hub is where the work is separated for each AI service.
For the purposes of this blog, you can create a Hub.
3. Fill in all of the appropriate information including the subscription youʼd like Azure AI Foundry deployed to and the resource group where it should exist.
For this demo project, you can leave the rest of the options as the default. However, in production, itʼs best to look at each option to ensure that youʼre deploying what the team(s) expects.
4. Once deployed, go to the Resource Group and youʼll see four (4) resources:
- The Hub.
- A storage account (everything you add to Azure Foundry AI shouldnʼt be ephemeral).
- The AI Services.
- Keyvault (store secrets).
Click the resource thatʼs the Azure AI services type.
5. Click the Go to Azure AI Foundry portal button.
6. Youʼre now in the AI Foundry portal.
7. To test out some features, click the blue + Create new deployment button and choose the From base models option.
8. Youʼll see that you have several Models to choose from, and the functionality of each ranges from chat competition to audio generation.
For the purposes of this demo, choose gpt-4o.
9. Click the blue Deploy button and feel free to left the name and deployment type as the default.
10. You now have your very own chat bot! This chat bot can also help you generate a prompt. For example, maybe you want to learn how to develop in Rust. You can ask the chatbot to generate a prompt for you thatʼll help the chatbot understand exactly what you need.
11. You can then run the prompt and see the output to help you on your Rust journey.
This is just one example of what you can use Azure AI Foundry for and itʼs encouraged to test it out yourself to see if thereʼs a particular business use case thatʼll help you on your AI journey. Just remember to delete your resources when youʼre done with your demo to ensure proper cost savings.
Closing Thoughts
Solutions like Azure AI Foundry allow engineers to build solutions around Agents and specific training to help them with their jobs and the production-level solutions that they want to implement. It helps business leaders and technical leaders get solutions out faster in a more repeatable and efficient fashion. An Amazing example of this are tools like Cursor AI, which is like having a free pair programmer to help you through building out and troubleshooting solutions.
GenAI, AI, Agents, or whatever else everyone is calling it is the next iteration in how tech is reinvented and how business can get to market faster while ensuring proper quality.
Frequently Asked Questions
What exactly is Azure AI Foundry, and how is it different from just using ChatGPT?
Azure AI Foundry is a managed service that gives you access to powerful AI models—like GPT-4o—but with more flexibility and control for enterprise use. Unlike ChatGPT, which is a general-purpose interface, AI Foundry lets you deploy, fine-tune, and integrate models into your own applications. You can customize the experience with your data, build AI Agents, and even tie everything into tools your teams already use, like GitHub or VS Code. It’s a more scalable, business-ready option for applied AI.
Do I need a background in data science or machine learning to use Azure AI Foundry?
Not at all. While a technical foundation helps, the platform is designed for engineers and developers who want to integrate AI into their workflows without having to build and train models from scratch. The UI is intuitive, and the setup process feels like working with other familiar Azure services. Microsoft has done a good job making AI deployment feel accessible, even for teams without deep AI expertise.
What are some real use cases for Azure AI Foundry in a business or engineering context?
You can use Azure AI Foundry to create internal chatbots, build code-generating agents, automate document analysis, or even train models on your company’s own data for better search and retrieval. Engineering teams might use it for pair programming, while business teams could deploy Agents to handle repetitive workflows. It’s a flexible platform that adapts to the specific needs of different teams, not a one-size-fits-all tool.
Is using Azure AI Foundry expensive?
It depends on your usage, but one of the key benefits is that you're not paying to host or train foundational models yourself—that's all handled by Microsoft. You only pay for what you use, and you can set budgets or delete unused resources to manage costs. For early experimentation or proof of concept projects, it’s very possible to stay within a modest budget.
How can I test Azure AI Foundry without committing to a big project?
The blog walks you through how to deploy a simple chatbot using GPT-4o in just a few steps. You can follow that exact process to experiment with different models, test prompt behavior, and even integrate your own data. Once you get comfortable, it’s easy to scale up or refine the use case. Just don’t forget to clean up your resources afterward to avoid unnecessary charges.
