Thursday, 21 January 2016

Matlab minimum for variables of different dimensions with the min() function

When using the Matlab function min() to find the minimum value of a variable, I wanted to use it for data that was in more than just 1 dimension. I received the following error message when running my code:

Error using min
Matrix dimensions must agree.

The problem here is that I was trying to find the minimum of several variables together, and the variables each had different dimensions.

Furthermore, the function acts somewhat differently to what I would expect. Consider the following initialisation:

>> data=[[1:10];[1:10]]
data =
     1     2     3     4     5     6     7     8     9    10
     1     2     3     4     5     6     7     8     9    10

Here the data fills a 2 x 10 matrix. I would expect the min() function to return one value for the entire matrix. Instead, the min() function returns the minimum for each column.

>>  min(data)
ans =
     1     2     3     4     5     6     7     8     9    10

One solution is to use multiple min() calls as follows:

>> min(min(data))
ans =
     1

But this isn't a great solution, as you might require this multiple times. So the solution is to use (:) to reference the elements of the matrix. This references all of the elements consecutively, and results in a 1 dimensional matrix containing all elements.

data(:)
ans =
     1
     1
     2
     2
     3
     3
     4
     4
     5
     5
     6
     6
     7
     7
     8
     8
     9
     9
    10
    10

As the data is now one dimensional, the min() function will work on the data in this format, as follows:

>> min(data(:))
ans =
     1

This is how I would have expected the min() function to behave by default, but at least this works for one variable. However if I now want to use it to find the minimum from multiple variables, again the behaviour starts to cause a problem. I will create two 2x10 matrices for testing as follows:

>>  a=[[1:10];[1:10]]
a =
     1     2     3     4     5     6     7     8     9    10
     1     2     3     4     5     6     7     8     9    10
>>  b=[[11:20];[11:20]]
b =
    11    12    13    14    15    16    17    18    19    20
    11    12    13    14    15    16    17    18    19    20

If trying to find the minimum by passing both variables to the min() function, this again doesn't work entirely as expected.

>>  min(a(:), b(:))
ans =
     1
     1
     2
     2
     3
     3
     4
     4
     5
     5
     6
     6
     7
     7
     8
     8
     9
     9
    10
    10

The reason for this is that Matlab sees the two variables as two columns of data that have been joined together, and then passed to the function. Matlab processes the data as follows:

>> [a(:),b(:)]
ans =
     1    11
     1    11
     2    12
     2    12
     3    13
     3    13
     4    14
     4    14
     5    15
     5    15
     6    16
     6    16
     7    17
     7    17
     8    18
     8    18
     9    19
     9    19
    10    20
    10    20

So we can see that the min() function actually finds which dimension is longest, and finds the minimum value in the shorter dimension. If we transpose the matrix first using the apostrophe, and then use the min function, the function is still operating on the longest dimension.

>> [a(:),b(:)]'
ans =
  Columns 1 through 18
     1     1     2     2     3     3     4     4     5     5     6     6     7     7     8     8     9     9
    11    11    12    12    13    13    14    14    15    15    16    16    17    17    18    18    19    19
  Columns 19 through 20
    10    10
    20    20
>> min([a(:),b(:)]')
ans =
  Columns 1 through 18
     1     1     2     2     3     3     4     4     5     5     6     6     7     7     8     8     9     9
  Columns 19 through 20
    10    10

There are a couple of obvious workarounds for this problem. The first would be to use multiple min() function calls, as follows:

>> min(min(a(:),min(b(:))))
ans =
     1

This isn't a particularly nice line of code, with three min() function calls. An alternative that I find better  can be created by appending the data before calling the min() function, and is as follows:

>> min([a(:); b(:)])
ans =
     1

Care must be taken to use the semicolon in between the matricies to minimum value, if using a column this will create two columns of data, and the min() function then returns two values as follows:

>>  min([a(:), b(:)])
ans =
     1    11

I'm not sure why in this case the min() function is now only returning two values, and not working in the longest dimension like it was previously.

Alternatively if using the comma to separate variables, the variables can be trtansposed first, in which case the function works, but this is more work for the same result and so isn't recommended.

>> min([a(:)', b(:)'])
ans =
     1

Summary

The behaviour of the min() function is to find the minimum value in a given matrix, not the absolute minimum in the data passed to the function. To get the function to behave in this manner requires some changing of the data format before the data is passed to the function. I believe the cleanest format is the following:

>> min(min(a(:),min(b(:))))
ans =
     1


Thursday, 31 December 2015

Correcting language reported by chrome to other websites

I was having many problems where Google Chrome would report the wrong language to websites that I was visiting, and I would get redirected to the foreign version of the website, not in English. I thought this was due to Chrome redirecting me to a foreign version of Google, but after fixing it so that I can always use the UK version, as mentioned in a previous post, I still had the problem of getting redirected to translated versions of the website. I thought this could be due to my location, and a geographical lookup of my IP, but once I returned to England I was still getting translations of websites I was visiting.

I tried a couple of times looking at the "Local State" file, but mine looked fine, and there was no mention of any foreign versions of Google.

I found the solution in a post at this link here: https://productforums.google.com/forum/#!topic/chrome/pKJ8eBCT4ec by CaseyD

To get to the options, the steps were slightly different, I clicked the "Customise and Control Google Chrome" button - the button with three horizontal lines in the top right corner of Chrome, the clicked the "Settings" option. Then you should be on the "Settings" tab, this is also accessible by entering chrome://settings/ into the address bar. From here, scroll to the bottom and click "+ Show Advanced Settings". Then scroll to the section "Languages", and click the button "Language and Input Settings...". In this list, drag up your preferred language to the top. The topmost language takes preference. Then click "Finished." and close the settings tab.

Wednesday, 30 December 2015

Connecting Python 2.7 to XAMPP MySQL Database on Windows

I installed the XAMPP package on my Windows 8.1 device, and wanted to use python 2.7 to write to the MySQL database. I found that there is the package "mysql-python" which should allow reading and writing to a MySQL DB from python, however was having problems when installing.

The error message was as follows:

    _mysql.c(42) : fatal error C1083: Cannot open include file: 'config-win.h':
No such file or directory
    error: command 'C:\\Users\\re\\AppData\\Local\\Programs\\Common\\Microsoft\\
Visual C++ for Python\\9.0\\VC\\Bin\\cl.exe' failed with exit status 2

    ----------------------------------------
Command "C:\Python27\python.exe -c "import setuptools, tokenize;__file__='c:\\users\\re\\appdata\\local\\temp\\pip-build-xnlpn9\\mysql-python\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\re\appdata\local\temp\pip-1k03uu-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\re\appdata\local\temp\pip-build-xnlpn9\mysql-python

I had tried to install the mysql python connector from this website: https://dev.mysql.com/downloads/connector/python/ however this didn't help. I'd found some posts saying that a file in the tar ball used by the pip install needed modifying, however installing the python connector above didn't seem to install any files in the directory as expected, so I wasn't able to update the tar ball to the directory that I needed.

I also tried updating the PATH and PYTHONPATH environmental variables with no luck.

The solution that worked for me eneded up being from this post on stackoverflow: http://stackoverflow.com/a/25445938

I downloaded the file from this website: http://www.lfd.uci.edu/~gohlke/pythonlibs/#mysql-python

I used the file MySQL_python-1.2.5-cp27-none-win32.whl and installed it using the following command:

pip install C:\MySQL_python-1.2.5-cp27-none-win32.whl

This finally installed successfully, and allowed me to the use the command "import mysql.connector" for my code.

Friday, 20 November 2015

Python importing compiled .pyc files in to the __main__ namespace

I had written a rather long piece of code in Python, which was all in one file, and all using the same namespace. I later needed to split the file and was hoping to find a method that worked by allowing me to "include" the file in a list, just like the code was in one large file.

Using import gave each module its own namespace, and so they weren't able to access the other imported modules without first importing the module again. Similarly, using any combination of from [name] import * had the same problem.

What ended up being the solution was the imp library.

The code that I have used is as follows:

import imp
imp.load_module(“__main__”, filename)

By passing "__main__", this loads the module into the workspace __main__, which is the namespace my code was using. I only used this to load .pyc files. The filename doesn't have to be a full path, and can just be the file name if located in the same folder as the script, assuming you haven't changed directories.

In hindsight, the problems were mostly from poor code structure, and each module should run as a standalone module, but due to time constraints I didn't have time to fix these issues.

Friday, 13 November 2015

Setting default search engine in Chrome to stop geographical redirection

I was somewhat frustrated when using Google Chrome when abroad, as it would always redirect to the local version of Google. The change in interface isn't the main problem, the problem is that localised versions of Google searches give localised results, which is more of a problem when you don't speak the language. There are options to change your Google account display settings, but this don't seem to always stick, and if you aren't logged in, or have multiple accounts then this setting doesn't help. Also there is the tip to use the "No Country Redirect" option, by accessing your prefered version of Google, followed by "/ncr", e.g. http://www.google.com/ncr however this only works if you set it as a bookmark or homepage, and not from the Omnibar.

Finally, I found a way to set the Omnibar to use my local Google. Go to chrome://settings/ and scroll down to the "Search" section. There, the deafault entry for Google will be as follows:

{google:baseURL}search?q=%s&{google:RLZ}... (clipped)

Here, changing the first bit of the string to the local Google isn't quite enough. I originally changed it to the following:

https://google.co.uk/search?q=%s&{google:RLZ}... (clipped)

However when closing the settings, it would revert back to {google:baseURL} which used to revert back to the local Google. By changing it to "https", this seems to have stopped it happening for now. I'm not sure why this is even a "feature", but seems to work for the moment at least. So the full search URL is as follows:

https://google.co.uk/search?q=%s&{google:RLZ}{google:originalQueryForSuggestion}{google:assistedQueryStats}{google:searchFieldtrialParameter}{google:bookmarkBarPinned}{google:searchClient}{google:sourceId}{google:instantExtendedEnabledParameter}{google:omniboxStartMarginParameter}ie={inputEncoding}

The following is a screenshot of how it looks now:



It also works to cut out all of the extra stuff, and so just make the string as follows:

https://google.co.uk/search?q=%s

If this was useful to you, I also have another post on how to correct the language reported by chrome to websites, which you can view here.

Matlab definition of functions, and function definition errors

I was trying to create a new function in Matlab, and had some issues as I am relatively unfamiliar with the format.

When writing a function, it must go in a .m file, and only the function can go in the .m file, no other code.  The typical syntax for a function

function output_variable=function_name(input_variable)

output_variable = 2* input_variable

end
The above function then should go in a file named function_name.m. Furthermore, to use the function, the code does not need to be executed first. The "current folder" must contain the .m script, and then just use the function name with variables if needed. Comments are also allowed.

If the function contains any code outside of the function, the following error will be seen:

This statement is not inside any function.
 (It follows the END that terminates the definition
 of the function "create_const_mat".)

In this case, remove any code before the first line defining the function name, or after the end line.

If the function is saved in a .m file with a different name from the function, the following error will be seen:

Error: Function definitions are not permitted in this
context.

To solve this error, rename the .m file to match the function name, or conversely rename the function name to match the .m file name.

If executing the code in the function, then there are other possible errors, which will depend on the variables in the current workspace, and whether they match the variables needed by the function. If, your function requires varaibles which are not defined, then expect the following error:

Not enough input arguments.

If they are defined incorrectly, then anything could happen.

Thursday, 30 July 2015

Reading a CSV file in Matlab with header rows and mixed text, timestamps, integers and doubles

I wanted to read in a CSV file in Matlab, but was having problems using csvread(), which only supports files with only numeric values. It seems as though the easiest way to read in a mixed CSV file, is to read in the header first, and then read in the data using textscan.

I based my code on an answer from StackExchange. For reading the header, the following code worked well for me:
fid = fopen(file, 'r');
Header = fgetl(fid);
fclose(fid);

Header = regexp(Header, '([^,]*)', 'tokens');
Header = cat(2, Header{:});
The values are then in cells, and we need to use the function cell2mat() to convert the data in to something we can work with. This works for both strings and numerical values.

Then to read the main body of the text file, I used textscan(). This requires a string defining the format. As my format string is quite long, due to having a lot of columns, I've defined the format string separately. I have used the following code to define the format:
format_a='%u%u%s%u%u%f%f%u%u';
format_b=repmat('%f',1,109); %repeat the string '%f' 109 times
format_c='%u%u%u';

format=strcat(format_a, format_b, format_c);
This generates a long string specifying the format of the file that will be read in. The input string '%u' is an unsigned integer, and the input string is for '%s' strings. I also has a time and date column in the CSV, which I read with the datenum() function later, so for now I have treated it as a string.

The reading in of the file is done with the textscan() function in the following few lines:
fid = fopen(file, 'r');D = textscan(fid, format, 'Delimiter', ',', 'HeaderLines', 1);fclose(fid);

By setting the Delimiter, this allows other options than only the comma. Also the number of header columns can be set by the number following 'HeaderLines'.

This reads all of the data in to the variable which is saved in cells. So to read in the numbers from the cells, I have used the function cell2mat(). Here is an example of how I read in one column:
index=cell2mat(D(1));
Similarly, to read in the data from multiple columns, I've used the following code:
 data=cell2mat(D(10:118));
 This works for integers and doubles. To handle the date, I have used the following line:
timestamps=datenum(D{3}, 'dd/mm/yyyy HH:MM');
This interprets the information in the strings, and creates a "datetime" object, which can be used much more easily when plotting data.