Microsoft Sentinel - Data Connectors and Ingestion

Summary

Microsoft Sentinel is only as good as the data it receives. Connectors bring logs in, data collection rules shape them on the way, tiers decide what each table costs and can do, and health monitoring tells you when a source goes quiet. This article walks through all four so a SOC can collect what matters and spot what is missing.

Detection Starts With Data

We will use A.M Finances Group as our business example. Its security team connected a new firewall cluster to Microsoft Sentinel on a Monday, and the firewall logged every allowed connection. By the end of the month, ingestion had doubled. Then an analyst hunting a suspicious sign-in found that the Linux log forwarder had been silent for nine days. One problem was too much data and the other was missing data, and both began at ingestion.

Microsoft Sentinel can only detect, hunt, and investigate across the data it has. Every analytics rule and workbook depends on logs arriving complete, in the right place, at a cost the business can sustain. Ingestion is where that chain begins.

Pick the Right Connector

Microsoft groups connectors by how the data travels. Service-to-service connectors use the Azure foundation to bring in Microsoft services such as Microsoft Defender XDR, along with Amazon Web Services. Agent-based connectors use the Azure Monitor Agent. Syslog via AMA and Common Event Format via AMA collect from Linux machines and from network and security appliances, usually through a log forwarder, and the data lands in the Syslog and CommonSecurityLog tables. For software-as-a-service and cloud sources, the Codeless Connector Framework polls a REST API, needs no servers to run, and includes health monitoring. Anything custom can send logs through the Logs Ingestion API.

You find and install most connectors from the Content hub, which packages each one with the workbooks, analytics rules, and playbooks that go with it. Stay off the old paths. The Log Analytics agent was retired on August 31, 2024, and support for the HTTP Data Collector API ended on September 14, 2026. A.M Finances Group sent its firewall's Common Event Format logs to a Linux forwarder running the Azure Monitor Agent.

Shape It Before It Lands

Data collection rules decide what arrives. A rule says which systems to collect from and which logs to take, and an ingestion-time transformation applies a query to the data before it is stored in the workspace. Microsoft lists four common uses: filter rows or columns to cut ingestion and storage costs, enrich records with extra columns, normalize logs to the Advanced Security Information Model, and mask personal information such as all but the last digits of a card number.

Not every connector supports transformations. The Azure Monitor Agent connectors do, as do the Logs Ingestion API and connectors built on the Codeless Connector Framework. Legacy codeless and Azure Functions connectors do not. In the Defender portal, filter and split rules, introduced in preview in April 2026, let you drop noise or send part of a table to the data lake without writing a rule by hand.

Choose the Right Tier

Every table lives in a tier. The analytics tier is the hot tier. It powers alerting, hunting, workbooks, and every Microsoft Sentinel feature, and analytics retention starts at 30 days and can extend up to two years. The data lake tier is a low-cost tier for high-volume logs. Its data is not available to real-time analytics or threat hunting, but total retention can reach 12 years, and you can still query it when an investigation needs history.

You set the tier for each table in the Defender portal, under Microsoft Sentinel, Configuration, and Tables. Choose with care, because moving a table from analytics to the data lake stops the alerting, analytics rules, and hunting that depend on it. A.M Finances Group used a split rule. Denied connections stay in the analytics tier for detection, and routine allowed connections go to the data lake only.

Watch the Pipeline

A connector that stops quietly is worse than one that fails loudly. Turn on auditing and health monitoring in the Defender portal so the SentinelHealth table records connector events. Install the Data collection health monitoring workbook from the Content hub to see volume, event rates, the time the last log was received, and agent heartbeat. Microsoft recommends Azure Monitor alert rules, rather than analytics rules, to alert on health drift.

That is how A.M Finances Group would have caught its nine silent days. The forwarder stopped sending, and nothing said so, because silence looked like a quiet network. The team now watches the last log received for every source.

Conclusion

Microsoft Sentinel starts with ingestion. Pick the connector that fits how the source talks, shape the data with a rule before it lands, place each table in the tier that matches its value, and watch health so silence never passes for safety. A.M Finances Group trimmed its noisy firewall feed, closed its nine-day gap, and now trusts the data its detections run on.