Aggregate publication counts in AuthorSearch().subject_areas

Author: nils-herrmann

docs/reference/scopus/AuthorSearch.rst

@@ -32,9 +32,8 @@ You can obtain a search summary just by printing the object:
 .. code-block:: python
 
     >>> print(s)
-    Search 'AUTHLAST(Selten) and AUTHFIRST(Reinhard)' yielded 2 authors as of 2024-05-11:
-		Selten, Reinhard; AUTHOR_ID:6602907525 (74 document(s))
-		Selten, Reinhard; AUTHOR_ID:57213632570 (1 document(s))
+    Search 'AUTHLAST(Selten) and AUTHFIRST(Reinhard)' yielded 1 author as of 2025-06-13:
+        Selten, Reinhard; AUTHOR_ID:6602907525 (76 document(s))
 
 
 To determine the the number of results use the `.get_results_size()` method, even before you download the results:
@@ -43,7 +42,7 @@ To determine the the number of results use the `.get_results_size()` method, eve
 
     >>> other = AuthorSearch("AUTHLAST(Selten)", download=False)
     >>> other.get_results_size()
-    27
+    30
 
 
 Primarily, the class provides a list of `namedtuples <https://docs.python.org/3/library/collections.html#collections.namedtuple>`_ storing author EIDs, which you can use for the :doc:`AuthorRetrieval <AuthorRetrieval>` class, and corresponding information:
@@ -54,27 +53,23 @@ Primarily, the class provides a list of `namedtuples <https://docs.python.org/3/
     [Author(eid='9-s2.0-6602907525', orcid=None, surname='Selten', initials='R.',
      givenname='Reinhard', affiliation='Universitat Bonn', documents=74,
      affiliation_id='60007493', city='Bonn', country='Germany',
-     areas='ECON (73); MATH (19); BUSI (16)')]
+     areas='ECON (72); BUSI (8)')]
 
 
+Please note that Scopus sometimes returns duplicate `areas`, which are then aggregated (e.g. 'ECON (51); ECON (21)' → 'ECON (72)').
+
 Working with namedtuples is straightforward: Using `pandas <https://pandas.pydata.org/>`_, you can quickly convert the results set into a DataFrame:
 
 .. code-block:: python
 
     >>> import pandas as pd
     >>> pd.set_option('display.max_columns', None)
     >>> print(pd.DataFrame(s.authors))
-                      eid orcid surname initials givenname  \
-    0   9-s2.0-6602907525  None Selten       R.  Reinhard
-    1  9-s2.0-57213632570  None Selten       R.  Reinhard
-
-                         affiliation  documents affiliation_id     city  country  \
-    0               Universität Bonn         74       60007493     Bonn  Germany   
-    1  Southwest Jiaotong University          1       60010421  Chengdu    China   
+                 eid orcid surname initials givenname       affiliation  \
+    0  9-s2.0-6602907525  None  Selten       R.  Reinhard  Universität Bonn   
 
-                                 areas  
-    0  ECON (73); MATH (19); BUSI (16)  
-    1                         COMP (3)
+    documents affiliation_id  city  country                areas  
+    0         76       60007493  Bonn  Germany  ECON (72); BUSI (8)
 
 
 Downloaded results are cached to expedite subsequent analyses.  This information may become outdated.  To refresh the cached results if they exist, set `refresh=True`, or provide an integer that will be interpreted as maximum allowed number of days since the last modification date.  For example, if you want to refresh all cached results older than 100 days, set `refresh=100`.  Use `ab.get_cache_file_mdate()` to obtain the date of last modification, and `ab.get_cache_file_age()` to determine the number of days since the last modification.

pybliometrics/scopus/author_search.py

@@ -3,7 +3,7 @@
 
 from pybliometrics.superclasses import Search
 from pybliometrics.utils import check_integrity, check_parameter_value, \
-    check_field_consistency, listify, make_search_summary
+    check_field_consistency, get_and_aggregate_subjects, listify, make_search_summary
 
 
 class AuthorSearch(Search):
@@ -15,7 +15,8 @@ def authors(self) -> Optional[list[namedtuple]]:
         documents affiliation affiliation_id city country areas)`.
 
         All entries are `str` or `None`.  Areas combines abbreviated subject
-        areas followed by the number of documents in this subject.
+        areas followed by the number of documents in this subject. The number of 
+        documents on duplicate subject areas is summed up.
 
         Raises
         ------
@@ -35,8 +36,8 @@ def authors(self) -> Optional[list[namedtuple]]:
             aff = item.get('affiliation-current', {})
             fields = item.get('subject-area',
                               [{'@abbrev': '', '@frequency': ''}])
-            areas = [f"{d.get('@abbrev', '')} ({d.get('@frequency', '')})"
-                     for d in listify(fields)]
+            subjects = get_and_aggregate_subjects(fields)
+            areas = [f"{abbrev} ({'' if freq == 0 else freq})" for abbrev, freq in subjects.items()]
             new = auth(eid=item.get('eid'),
                        orcid=item.get('orcid'),
                        initials=name.get('initials'),

pybliometrics/scopus/tests/test_AuthorSearch.py

@@ -19,7 +19,7 @@ def test_authors():
     expected = Author(eid='9-s2.0-6602907525', orcid=None, surname='Selten',
         initials='R.', givenname='Reinhard', affiliation='Universität Bonn',
         documents=76, affiliation_id='60007493', city='Bonn',
-        country='Germany', areas='ECON (78); MATH (21); BUSI (16)')
+        country='Germany', areas='ECON (72); BUSI (8)')
     assert s1.authors[0] == expected
 
 

pybliometrics/utils/parse_content.py

@@ -64,6 +64,19 @@ def deduplicate(lst):
                  [])
     return new
 
+def get_and_aggregate_subjects(fields):
+    """Get and aggregate subject areas from Scopus AuthorSearch."""
+    frequencies = {}
+    for field in fields:
+        abbrev = field.get('@abbrev', '')
+        freq_str = field.get('@frequency', '')
+        frequency = int(freq_str) if freq_str.isdigit() else 0
+        if abbrev in frequencies:
+            frequencies[abbrev] += frequency
+        else:
+            frequencies[abbrev] = frequency
+    return frequencies
+
 
 def get_id(s, integer=True):
     """Helper function to return the Scopus ID at a fixed position."""