Mobius icon indicating copy to clipboard operation
Mobius copied to clipboard

RegisterTempTable not found by compiler

Open kvansaders opened this issue 7 years ago • 0 comments

After attempting to copy the sample code from the readme, I get a compilation error when trying to call the RegisterTempTable method on a data frame.


var sparkConf = new SparkConf();

sparkConf.SetMaster("local[2]"); //local standalone with 2 cores, ex:"spark://master:7077" to connect to other cluster
sparkConf.SetAppName("SparkCLRExample");
//sparkConf.SetSparkHome(""); //used to set the spark install location on worker nodes.  Using default instead.

SparkContext sc = SparkContext.GetOrCreate(sparkConf);


var reqDataFrame = sc.TextFile(@"hdfs://path/to/requests.csv");
var metricDataFrame = sc.TextFile(@"hdfs://path/to/metrics.csv");
reqDataFrame.RegisterTempTable("requests");
metricDataFrame.RegisterTempTable("metrics");
// C0 - guid in requests DataFrame, C3 - guid in metrics DataFrame  
var joinDataFrame = GetSqlContext().Sql(
	"SELECT joinedtable.datacenter" +
		 ", MAX(joinedtable.latency) maxlatency" +
		 ", AVG(joinedtable.latency) avglatency " +
	"FROM (" +
	   "SELECT a.C1 as datacenter, b.C6 as latency " +
	   "FROM requests a JOIN metrics b ON a.C0  = b.C3) joinedtable " +
	"GROUP BY datacenter");
joinDataFrame.ShowSchema();
joinDataFrame.Show();

'RDD' does not contain a definition for 'RegisterTempTable' and no extension method 'RegisterTempTable' accepting a first argument of type 'RDD' could be found (press F4 to add a using directive or assembly reference)

kvansaders avatar Feb 21 '18 14:02 kvansaders