Norway?

[bluetyson]

Anecdotally, 'norway' seems to be popular. Even for example when looking at SA Exploration reports. Or here:-
This was from a proposal to do predictive analytics on a bunch of geoscience stuff with some dubious analogies - from a Canadian company.

"sti_keywords": [
[
{
"keyword": "education",
"probability": 0.9999927878379822,
"unstemmed": "EDUCATION"
},
{
"keyword": "prediction",
"probability": 0.9999526143074036,
"unstemmed": "PREDICTIONS"
},
{
"keyword": "stability",
"probability": 0.9999324679374695,
"unstemmed": "STABILITY"
},
{
"keyword": "frequency",
"probability": 0.9998124837875366,
"unstemmed": "FREQUENCIES"
},
{
"keyword": "simulation",
"probability": 0.9997246265411377,
"unstemmed": "SIMULATION"
},
{
"keyword": "detection",
"probability": 0.9996479153633118,
"unstemmed": "DETECTION"
},
{
"keyword": "organization",
"probability": 0.999604344367981,
"unstemmed": "ORGANIZATIONS"
},
{
"keyword": "information system",
"probability": 0.9994807243347168,
"unstemmed": "INFORMATION SYSTEMS"
},
{
"keyword": "test",
"probability": 0.9952819347381592,
"unstemmed": "TESTS"
},
{
"keyword": "management method",
"probability": 0.9947834014892578,
"unstemmed": "MANAGEMENT METHODS"
},
{
"keyword": "document",
"probability": 0.9927297234535217,
"unstemmed": "DOCUMENTS"
},
{
"keyword": "sensitivity analysi",
"probability": 0.9906492829322815,
"unstemmed": "SENSITIVITY ANALYSIS"
},
{
"keyword": "survey",
"probability": 0.9885730743408203,
"unstemmed": "SURVEYS"
},
{
"keyword": "measurement",
"probability": 0.9878258109092712,
"unstemmed": "MEASUREMENT"
},
{
"keyword": "on-line system",
"probability": 0.9819966554641724,
"unstemmed": "ON-LINE SYSTEMS"
},
{
"keyword": "probability theory",
"probability": 0.9808198809623718,
"unstemmed": "PROBABILITY THEORY"
},
{
"keyword": "decision making",
"probability": 0.9807706475257874,
"unstemmed": "DECISION MAKING"
},
{
"keyword": "statistical analysi",
"probability": 0.9802524447441101,
"unstemmed": "STATISTICAL ANALYSIS"
},
{
"keyword": "commerce",
"probability": 0.9727779626846313,
"unstemmed": "COMMERCE"
},
{
"keyword": "wind (meteorology)",
"probability": 0.9680668115615845,
"unstemmed": "WIND (METEOROLOGY)"
},
{
"keyword": "ranking",
"probability": 0.966843843460083,
"unstemmed": "RANKING"
},
{
"keyword": "norway",
"probability": 0.9654883742332458,
"unstemmed": "NORWAY"
},
{
"keyword": "turbulence",
"probability": 0.9554965496063232,
"unstemmed": "TURBULENCE"
},
{
"keyword": "environment",
"probability": 0.9523953795433044,
"unstemmed": "ENVIRONMENTS"
},
{
"keyword": "artificial intelligence",
"probability": 0.9461550712585449,
"unstemmed": "ARTIFICIAL INTELLIGENCE"
},
{
"keyword": "spectrum",
"probability": 0.9431440830230713,
"unstemmed": "SPECTRA"
},
{
"keyword": "supplying",
"probability": 0.9292123317718506,
"unstemmed": "SUPPLYING"
},
{
"keyword": "thesis",
"probability": 0.9177210330963135,
"unstemmed": "THESES"
},
{
"keyword": "industry",
"probability": 0.8984866738319397,
"unstemmed": "INDUSTRIES"
},
{
"keyword": "research management",
"probability": 0.8902236223220825,
"unstemmed": "RESEARCH MANAGEMENT"
},
{
"keyword": "logistic",
"probability": 0.8653072118759155,
"unstemmed": "LOGISTICS"
},
{
"keyword": "marketing",
"probability": 0.8555718660354614,
"unstemmed": "MARKETING"
},
{
"keyword": "material",
"probability": 0.8554256558418274,
"unstemmed": "MATERIALS"
},
{
"keyword": "exploration",
"probability": 0.8326903581619263,
"unstemmed": "EXPLORATION"
},
{
"keyword": "finance",
"probability": 0.7999643683433533,
"unstemmed": "FINANCE"
},
{
"keyword": "procedure",
"probability": 0.7985092997550964,
"unstemmed": "PROCEDURES"
},
{
"keyword": "plasmas (physics)",
"probability": 0.7585477828979492,
"unstemmed": "PLASMAS (PHYSICS)"
},
{
"keyword": "management system",
"probability": 0.7525011897087097,
"unstemmed": "MANAGEMENT SYSTEMS"
},
{
"keyword": "personnel",
"probability": 0.7381218075752258,
"unstemmed": "PERSONNEL"
},
{
"keyword": "quality control",
"probability": 0.6919220685958862,
"unstemmed": "QUALITY CONTROL"
},
{
"keyword": "binders (materials)",
"probability": 0.682461678981781,
"unstemmed": "BINDERS (MATERIALS)"
},
{
"keyword": "statistic",
"probability": 0.6598255634307861,
"unstemmed": "STATISTICS"
},
{
"keyword": "technology transfer",
"probability": 0.6495755910873413,
"unstemmed": "TECHNOLOGY TRANSFER"
},
{
"keyword": "economic",
"probability": 0.6489657163619995,
"unstemmed": "ECONOMICS"
},
{
"keyword": "environment management",
"probability": 0.6489007472991943,
"unstemmed": "ENVIRONMENT MANAGEMENT"
},
{
"keyword": "reliability",
"probability": 0.6394709348678589,
"unstemmed": "RELIABILITY"
},
{
"keyword": "methodology",
"probability": 0.5842968225479126,
"unstemmed": "METHODOLOGY"
},
{
"keyword": "life (durability)",
"probability": 0.560967743396759,
"unstemmed": "LIFE (DURABILITY)"
},
{
"keyword": "task",
"probability": 0.5498309135437012,
"unstemmed": "TASKS"
},
{
"keyword": "feedback",
"probability": 0.5493220686912537,
"unstemmed": "FEEDBACK"
},
{
"keyword": "embedding",
"probability": 0.5298750996589661,
"unstemmed": "EMBEDDING"
},
{
"keyword": "model",
"probability": 0.5171043276786804,
"unstemmed": "MODELS"
},
{
"keyword": "image",
"probability": 0.5064711570739746,
"unstemmed": "IMAGES"
},
{
"keyword": "market research",
"probability": 0.5063830018043518,
"unstemmed": "MARKET RESEARCH"
}
]
],
"topic_probabilities": [
[
{
"keyword": "social and information science",
"probability": 0.9999999967051496,
"unstemmed": "social and information sciences"
},
{
"keyword": "life science",
"probability": 0.9997857836273429,
"unstemmed": "life sciences"
},
{
"keyword": "mathematical and computer science",
"probability": 0.9995673040968835,
"unstemmed": "mathematical and computer sciences"
},
{
"keyword": "geoscience",
"probability": 0.6036738988592714,
"unstemmed": "geosciences"
},
{
"keyword": "aeronautic",
"probability": 0.006360614090616023,
"unstemmed": "aeronautics"
},
{
"keyword": "astronautic",
"probability": 0.0010483669327786213,
"unstemmed": "astronautics"
},
{
"keyword": "engineering",
"probability": 0.0006828873398622611,
"unstemmed": "engineering"
},
{
"keyword": "chemistry and material",
"probability": 0.0006251699807542186,
"unstemmed": "chemistry and materials"
},
{
"keyword": "space science",
"probability": 0.0004059099588450474,
"unstemmed": "space sciences"
},
{
"keyword": "physic",
"probability": 0.000008485140273070631,
"unstemmed": "physics"
},
{
"keyword": "general",
"probability": 4.569450859931134e-8,
"unstemmed": "general"
}
]
],
"topic_threshold": 1

[abuonomo]

That is strange. Not sure why Norway shows up. There is no mention of Norway in the report? Also, how long is the report? One thing to note is these models were trained on abstracts. The performance tends to degrade a bit with very long texts. We actually found that the performance nearly matched that of an abstract when we automatically summarized longer texts down to abstract length (for example, using sumy).

[bluetyson]

No, and seen it in quite a few others...so relating Norway to something. So summarise and keyword the summary, as well?

[bluetyson]

Sumy link says not found btw.

[JustinGOSSES]

https://github.com/miso-belica/sumy

[JustinGOSSES]

I think Anthony was suggesting to use 'sumy' to summarize large documents down to abstract length.

It is weird that "Norway" is one of the predicted tags. Are there place names, proper names, or acronyms in the text that could relate to Norway?

[bluetyson]

Didn't look like it and I am somewhat familiar with Norwegian and place names. Came up some times in some purely South Australian things too, The above had a Canadian address in it. Might have to run it on some chunks to narrow it down.

[bluetyson]

That library does look handy, thanks.

[bluetyson]

Checked a few of them... maybe 20% of docs had it come up in this 600 odd sample. Checked a few just then for the existence of norway - not that, so something related.

[bluetyson]

Took the example above and kept shrinking it to find the 'Norwayish' bit. When doing so the Norway probability declined finally down to 0.018 - shrunk it further, then had 0.025 Sweden. Only proper nouns were two company names and SOW as an acronym. So interesting.

[abuonomo]

I'm not sure what the particular reason is for Norway appearing. I think we would have to manually inspect the model for that individual word and see how it is weighting term importances. Could be a case of overfitting. When you say shrink, do you mean you are using automated summarization?

[JustinGOSSES]

Just saw this tool, might be useful for debugging this type of problem https://github.com/pair-code/lit

[bluetyson]

Anthony, no, taking sections of the text to try and see which indicated Norway. Justin, saw that one yesterday, will have to investigate. Maybe a higher proportion of abstracts mentioning norway are exploration geosciencey than others, so a longer document like that gets more similarities.

[bluetyson]

Tomorrow should be able to do a 'Norway' count on a fair set of documents.

[bluetyson]

7000 odd reports (some from mangled OCR).

Norway keyword percentage: 82.1

[abuonomo]

Interesting. There must be a really strong correlation between the type of content you're feeding in and Norway in the training data. I'll try to take a closer look at the Norway model at some point and see if it leads to any insights.