Apache Spark / Big Data Administrator
Envision Technology Solutions · Berkeley Heights, New Jersey, United States
FULL TIMEpermanent
Job Description
Hi,
Experience, qualification, and soft skills, have you got everything required to succeed in this opportunity Find out below.
Hope you are doing well.
We have the below requirement with my client, please let me know if you are looking for a job.
Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJWork Type: 5 days a week at client office
Job Description10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loadsExcellent coordination with stakeholders and communication skills
Must have experience with below responsibilitiesCluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads. Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster. xzbcwsv Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs
Saurabh Kumar Team Lead – Recruitment Envision Technology SolutionsDirect Number Email:
Experience, qualification, and soft skills, have you got everything required to succeed in this opportunity Find out below.
Hope you are doing well.
We have the below requirement with my client, please let me know if you are looking for a job.
Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJWork Type: 5 days a week at client office
Job Description10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loadsExcellent coordination with stakeholders and communication skills
Must have experience with below responsibilitiesCluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads. Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster. xzbcwsv Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs
Saurabh Kumar Team Lead – Recruitment Envision Technology SolutionsDirect Number Email:
Details
| Company | Envision Technology Solutions |
| Location | Berkeley Heights, New Jersey, United States |
| Type | FULL TIME |
| Niche | tech |
| Experience | permanent |