Question: I am quite new to MySQL. I have data in more than 2000 text files that I need to import into a table. I have created the table as follow: 

```
CREATE TABLE test1
(   
    Month TINYINT UNSIGNED,
    Day TINYINT UNSIGNED,
    Year TINYINT UNSIGNED,
    Weight FLOAT UNSIGNED,
    length FLOAT UNSIGNED,
    Site_Number CHAR(3),
    id INT UNSIGNED NOT NULL AUTO_INCREMENT,
    PRIMARY KEY (id)
);

```

All data in my text files are comma separated. Each file contains thousands of rows and each row include month, day, year, weight and length information. The site\_number is the first three numbers of the file name. For example, if the file name is A308102316.txt, the site\_number of all the records imported from this file will be 308. Also, I want the ID to be auto increment. 

My question is: how can I achieve what I described above using MySQL command line client as I know nothing about PHP or other. I know how to import one text file into the table, but don't know how to write a loop to read many of them. Also, not know how to get site\_number from the file name.

Thank you for any help that you can provide.

Answer: 
Pattern match is performed linearly as it appears in the sources, so, this line:

```
case Row(t1: Seq[String], t2: Seq[String]) => Some((t1 zip t2).toMap)

```

Which doesn't have any restrictions on the values of t1 and t2 never matter match with the null value.

Effectively, put the null check before and it should work.
I think you will need to encode null values to blank or special String before performing assert operations. Also keep in mind that Spark executes lazily. So from the like "val values = df.flatMap" onward everything is executed only when show() is executed.