Options
All
  • Public
  • Public/Protected
  • All
Menu

Class SQLCluster

Hierarchy

  • SQLCluster

Index

Accessors

Methods

Accessors

context

Methods

createCSVTable

  • createCSVTable(tableName: string, filePaths: string[]): Promise<void>

  • Create a SQL table from CSV file(s).

    example
    import {sqlCluster} from '@rapidsai/sql';

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createCSVTable('test_table', ['test.csv']);

    Parameters

    • tableName: string

      Name of the table when referenced in a query

    • filePaths: string[]

      array of paths to CSV file(s)

    Returns Promise<void>

createDataFrameTable

  • createDataFrameTable(tableName: string, input: DataFrame<any>): Promise<void>

  • Create a SQL table to be used for future queries.

    example
    import {Series, DataFrame, Int32} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new({type: new Int32(), data: [1, 2, 3]});
    const b  = Series.new({type: new Int32(), data: [4, 5, 6]});
    const df = new DataFrame({'a': a, 'b': b});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);

    Parameters

    • tableName: string

      Name of the table when referenced in a query

    • input: DataFrame<any>

      DataFrame or paths to CSV files

    Returns Promise<void>

createORCTable

  • createORCTable(tableName: string, filePaths: string[]): Promise<void>

  • Create a SQL table from Apache ORC file(s).

    example
    import {sqlCluster} from '@rapidsai/sql';

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createORCTable('test_table', ['test.orc']);

    Parameters

    • tableName: string

      Name of the table when referenced in a query

    • filePaths: string[]

      array of paths to ORC file(s)

    Returns Promise<void>

createParquetTable

  • createParquetTable(tableName: string, filePaths: string[]): Promise<void>

  • Create a SQL table from Apache Parquet file(s).

    example
    import {sqlCluster} from '@rapidsai/sql';

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createParquetTable('test_table', ['test.parquet']);

    Parameters

    • tableName: string

      Name of the table when referenced in a query

    • filePaths: string[]

      array of paths to Parquet file(s)

    Returns Promise<void>

describeTable

  • describeTable(tableName: string): Map<string, DataType>

  • Returns a map with column names as keys and the column data type as values.

    example
    import {Series, DataFrame, Int32} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new({type: new Int32(), data: [1, 2, 3]});
    const df = new DataFrame({'a': a});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);
    console.log(sqlCluster.describeTable('test_table'));
    // {'a': Int32}

    Parameters

    • tableName: string

    Returns Map<string, DataType>

dropTable

  • dropTable(tableName: string): Promise<void>

  • Drop a SQL table from SQLContext memory.

    example
    import {Series, DataFrame, Int32} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new({type: new Int32(), data: [1, 2, 3]});
    const b  = Series.new({type: new Int32(), data: [4, 5, 6]});
    const df = new DataFrame({'a': a, 'b': b});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);
    await sqlCluster.dropTable('test_table');
    console.log(await sqlCluster.listTables());
    // []

    Parameters

    • tableName: string

      Name of the table to drop

    Returns Promise<void>

explain

  • explain(sql: string, detail?: boolean): string

  • Returns a break down of a given query's logical relational algebra plan.

    example
    import {Series, DataFrame} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new([1, 2, 3]);
    const df = new DataFrame({'a': a});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);

    console.log(sqlCluster.explain('SELECT a FROM test_table'));
    // BindableTableScan(table=[[main, test_table]], aliases=<a href="cuml_src.UMAP.html#a">a</a>)

    Parameters

    • sql: string

      SQL query

    • detail: boolean = false

      if a physical plan should be returned instead

    Returns string

kill

  • kill(): void

  • Sends a SIGTERM signal to all spawned workers. Essentially terminates all spawned workers and removes any references to them.

    example
    import {SQLCluster} from '@rapidsai/sql';

    const sqlCluster = await SQLCluster.init();
    sqlCluster.kill();

    Returns void

listTables

  • listTables(): string[]

  • Returns an array with the names of all created tables.

    example
    import {Series, DataFrame, Int32} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new({type: new Int32(), data: [1, 2, 3]});
    const df = new DataFrame({'a': a});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);
    console.log(await sqlCluster.listTables());
    // ['test_table']

    Returns string[]

sql

  • sql(query: string): AsyncGenerator<DataFrame<any>, void, undefined>

  • Query a SQL table and return the result as a DataFrame.

    example
    import {Series, DataFrame, Int32} from '@rapidsai/cudf';
    import {SQLCluster} from '@rapidsai/sql';

    const a  = Series.new({type: new Int32(), data: [1, 2, 3]});
    const b  = Series.new({type: new Int32(), data: [4, 5, 6]});
    const df = new DataFrame({'a': a, 'b': b});

    const sqlCluster = await SQLCluster.init();
    await sqlCluster.createTable('test_table', df);

    for await (const df of sqlCluster.sql('SELECT a FROM test_table')) {
      console.log(df.toString());
    }
    //  a
    //  0
    //  1
    //  2
    //  3

    Parameters

    • query: string

      SQL query string

    Returns AsyncGenerator<DataFrame<any>, void, undefined>

Static init

  • init(options?: Partial<ClusterProps>): Promise<SQLCluster>

  • Initialize and return a new pool of SQLCluster workers.

    example
    import {SQLCluster} from '@rapidsai/sql';

    const cluster = await Cluster.init();

    Parameters

    • options: Partial<ClusterProps> = {}

      options for the SQLCluster and SQLContext instance(s)

    Returns Promise<SQLCluster>