spark运行错误解决

spark运行错误解决

问题：”java.lang.IllegalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-memory option or in Spark configuration.
解决；Vmoptions中添加-=1073741824

问题：用spark连接mysql出现Exception in thread “main” java.lang.ClassNotFoundException: sql.jdbc.Driver

解决：idea中resource中引入mysql驱动包并use as library添加path中

问题：java.lang.NoSuchMethodError: org.apache.hadoop.LocationInfo()[Lorg/apache/hadoop/mapred/SplitLocationInfo;
由于缺少下面的依赖包

<dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-mapreduce-client-core</artifactId><version>2.7.3</version>
<dependency>

Error:(48, 45) value toDS is not a member of Seq[Person]
val caseClassDS = Seq(Person(“asd”,43)).toDS()错误
将case class Person()移到main方法外