Principal Site Reliability Engineer

M&T Bank · United States

Overview: Responsible for designing, implementing, and continuously improving highly reliable, scalable, and resilient platform solutions across the enterprise. Operates as a subject matter expert (SME) in Site Reliability Engineering, driving reliability engineering practices, operational excellence, and automation across the Software Development Lifecycle. Leads complex initiatives, influences enterprise engineering standards, and partners with senior stakeholders to improve system stability, observability, and performance. Serves as a mentor and technical leader for less experienced engineers across Technology. Primary Responsibilities:

• Accountable for defining and driving service reliability standards, including SLOs, SLAs, and error budgets across platforms.
• Design and implement highly available, fault-tolerant architectures aligned with enterprise scalability and resiliency requirements.
• Lead incident management practices, including detection, response, escalation, and recovery processes.
• Drive problem management and root cause analysis to prevent systemic issues.
• Develop and promote observability strategies, including logging, monitoring, alerting, and tracing.
• Lead automation initiatives for self-healing systems and operational workflows.
• Contribute to and review technical roadmaps with reliability and performance considerations.
• Partner with development, infrastructure, cybersecurity, and architecture teams.
• Serve as a technical authority for performance, resilience, and capacity planning.
• Drive production readiness practices including performance testing and failover capabilities.
• Lead cross-team reliability improvement initiatives.
• Participate in and lead post-incident reviews ensuring actionable outcomes.
• Mentor engineers on reliability engineering and best practices.
• Engage with stakeholders to identify risks and optimization opportunities.
• Ensure adherence to risk and regulatory standards and escalate issues when needed.
• Maintain internal control standards and compliance expectations.